如何在Android实现录屏直播
許多開發者在做智慧教室同屏亦或會議同屏時,基于Andriod平臺采集屏幕并編碼推送,往往遇到各種各樣的問題,以下就我們開發過程中的一些技術考量做個分享,權當拋磚引玉:
協議選擇、數據來源和處理
1. 內網環境下,組播還是RTMP?
回答:這個問題,被無數的開發者問到,為此,單獨寫了篇博客論證:https://blog.csdn.net/renhui1112/article/details/86741428,感興趣的可以參考下,簡單來說,能RTMP的,就RTMP,如果真是內網環境下,沒有并發瓶頸的同屏,可以啟動內置RTSP服務(走單播),然后,其他終端拉流也不失為一個好的方案。
2. 推送分辨率如何設定或縮放?
回答:一般來說,好多Android設備,特別是高分屏,拿到的視頻原始寬高非常大,如果推原始分辨率,編碼和上行壓力大,所以,一般建議,適當縮放,比如寬高縮放至2/3,縮放一般建議等比例縮放,此外,縮放寬高建議16字節對齊。
廢話不多說,上實例代碼:
private void createScreenEnvironment() {sreenWindowWidth = mWindowManager.getDefaultDisplay().getWidth();screenWindowHeight = mWindowManager.getDefaultDisplay().getHeight();Log.i(TAG, "screenWindowWidth: " + sreenWindowWidth + ",screenWindowHeight: "+ screenWindowHeight);if (sreenWindowWidth > 800){if (screenResolution == SCREEN_RESOLUTION_STANDARD){scale_rate = SCALE_RATE_HALF;sreenWindowWidth = align(sreenWindowWidth / 2, 16);screenWindowHeight = align(screenWindowHeight / 2, 16);}else if(screenResolution == SCREEN_RESOLUTION_LOW){scale_rate = SCALE_RATE_TWO_FIFTHS;sreenWindowWidth = align(sreenWindowWidth * 2 / 5, 16);}}Log.i(TAG, "After adjust mWindowWidth: " + sreenWindowWidth + ", mWindowHeight: " + screenWindowHeight);int pf = mWindowManager.getDefaultDisplay().getPixelFormat();Log.i(TAG, "display format:" + pf);DisplayMetrics displayMetrics = new DisplayMetrics();mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);mScreenDensity = displayMetrics.densityDpi;mImageReader = ImageReader.newInstance(sreenWindowWidth,screenWindowHeight, 0x1, 6);mMediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);}3. 橫豎屏自動適配
回答:因為橫豎屏狀態下,采集的屏幕寬高不一樣,如果橫豎屏切換,這個時候,需要考慮到橫豎屏適配問題,確保比如豎屏狀態下,切換到橫屏時,推拉流兩端可以自動適配,橫豎屏自動適配,編碼器需要重啟,拉流端,需要能自動適配寬高變化,自動播放。
4. 一定的補幀策略
回答:好多人不理解為什么要補幀,實際上,屏幕采集的時候,屏幕不動的話,不會一直有數據下去,這個時候,比較好的做法是,保存最后一幀數據,設定一定的補幀間隔,確保不會因為幀間距太大,導致播放端幾秒都收不到數據,當然,如果服務器可以緩存GOP,這個問題迎刃而解。
5. 異常網絡處理、事件回調機制
回答:如果是走RTMP,網絡抖動或者其他網絡異常,需要有好重連機制和狀態回饋機制。
class EventHandeV2 implements NTSmartEventCallbackV2 {@Overridepublic void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);String publisher_event = "";switch (id) {case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:publisher_event = "開始..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:publisher_event = "連接中..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:publisher_event = "連接失敗..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:publisher_event = "連接成功..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:publisher_event = "連接斷開..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:publisher_event = "關閉..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:publisher_event = "開始一個新的錄像文件 : " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:publisher_event = "已生成一個錄像文件 : " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:publisher_event = "發送時延: " + param1 + " 幀數:" + param2;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:publisher_event = "快照: " + param1 + " 路徑:" + param3;if (param1 == 0) {publisher_event = publisher_event + "截取快照成功..";} else {publisher_event = publisher_event + "截取快照失敗..";}break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:publisher_event = "RTSP服務URL: " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:publisher_event ="服務器不支持RTSP推送, 推送的RTSP URL: " + param3;break;}String str = "當前回調狀態:" + publisher_event;Log.i(TAG, str);Message message = new Message();message.what = PUBLISHER_EVENT_MSG;message.obj = publisher_event;handler.sendMessage(message);}}6. 部分屏幕數據采集
回答:我們遇到的好多場景下,教室端,會拿出來3/4的區域用來投遞給學生看,1/4的區域,用來做一些指令等操作,這個時候,就需要考慮屏幕區域裁剪,接口可做如下設計:
/*** 投遞裁剪過的RGBA數據** @param data: RGBA data** @param rowStride: stride information** @param width: width** @param height: height** @param clipedLeft: 左; clipedTop: 上; clipedwidth: 裁剪后的寬; clipedHeight: 裁剪后的高; 確保傳下去裁剪后的寬、高均為偶數** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight); //實際裁剪比例,可酌情自行調整int left = 100;int cliped_left = 0;int top = 0;int cliped_top = 0;int cliped_width = width_;int cliped_height = height_;if(scale_rate == SCALE_RATE_HALF){cliped_left = left / 2;cliped_top = top / 2;//寬度裁剪后,展示3/4比例cliped_width = (width_ *3)/4;//高度不做裁剪cliped_height = height_;}else if(scale_rate == SCALE_RATE_TWO_FIFTHS){cliped_left = left * 2 / 5;cliped_top = top * 2 / 5;//寬度裁剪后,展示3/4比例cliped_width = (width_ *3)/4;//高度不做裁剪cliped_height = height_;}if(cliped_width % 2 != 0){cliped_width = cliped_width + 1;}if(cliped_height % 2 != 0){cliped_height = cliped_height + 1;}if ( (cliped_left + cliped_width) > width_){Log.e(TAG, " invalid cliped region settings, cliped_left: " + cliped_left + " cliped_width:" + cliped_width + " width:" + width_);return;}if ( (cliped_top + cliped_height) > height_){Log.e(TAG, "invalid cliped region settings, cliped_top: " + cliped_top + " cliped_height:" + cliped_height + " height:" + height_);return;}//Log.i(TAG, " clipLeft: " + cliped_left + " clipTop: " + cliped_top + " clipWidth: " + cliped_width + " clipHeight: " + cliped_height);libPublisher.SmartPublisherOnCaptureVideoClipedRGBAData(publisherHandle, last_buffer, row_stride_,width_, height_, cliped_left, cliped_top, cliped_width, cliped_height );7. 文字、圖片水印
回答:好多場景下,同屏者會把公司logo,和一定的文字信息展示在推送端,這個時候,需要考慮到文字和圖片水印問題,具體可參考如下接口設置:
/*** Set Text water-mark(設置文字水印)* * @param fontSize: it should be "MEDIUM", "SMALL", "BIG"* * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".* * @param xPading, yPading: the distance of the original picture.* * <pre> The interface is only used for setting font water-mark when publishing stream. </pre> * * @return {0} if successful*/public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);/*** Set Text water-mark font file name(設置文字水印字體路徑)** @param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf** @return {0} if successful*/public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);/*** Set picture water-mark(設置png圖片水印)* * @param picPath: the picture working path, e.g: /sdcard/logo.png* * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".* * @param picWidth, picHeight: picture width & height* * @param xPading, yPading: the distance of the original picture.* * <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre> * * @return {0} if successful*/public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);如何通過RTMP或RTSP協議發送屏幕數據
RTMP或RTSP方案,有很多開源的選擇,不過大多是demo級,如果產品化,自(填)研(坑)需要很大的精力,以下以大牛直播SDK為例,介紹下如何對接數據,好多開發者提到,為什么大牛直播SDK的Android平臺RTMP推送接口怎么這么多?不像一些開源或者商業RTMP推送一樣,就幾個接口,簡單明了。廢話不多說,感興趣的可以參考下?Github:
1. 初始化Publisher接口,返回推送實例句柄:
/*** Open publisher(啟動推送實例)** @param ctx: get by this.getApplicationContext()* * @param audio_opt:* if 0: 不推送音頻* if 1: 推送編碼前音頻(PCM)* if 2: 推送編碼后音頻(aac/pcma/pcmu/speex).* * @param video_opt:* if 0: 不推送視頻* if 1: 推送編碼前視頻(YUV420SP/YUV420P/RGBA/ARGB)* if 2: 推送編碼后視頻(H.264)** @param width: capture width; height: capture height.** <pre>This function must be called firstly.</pre>** @return the handle of publisher instance*/public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt, int width, int height);2. 相關Event事件回調,如網絡狀態、實時快照、錄像狀態等回調:
/*** Set callback event(設置事件回調)* * @param callbackv2: callback function* * @return {0} if successful*/public native int SetSmartPublisherEventCallbackV2(long handle, NTSmartEventCallbackV2 callbackv2); class EventHandeV2 implements NTSmartEventCallbackV2 {@Overridepublic void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);String publisher_event = "";switch (id) {case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:publisher_event = "開始..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:publisher_event = "連接中..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:publisher_event = "連接失敗..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:publisher_event = "連接成功..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:publisher_event = "連接斷開..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:publisher_event = "關閉..";break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:publisher_event = "開始一個新的錄像文件 : " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:publisher_event = "已生成一個錄像文件 : " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:publisher_event = "發送時延: " + param1 + " 幀數:" + param2;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:publisher_event = "快照: " + param1 + " 路徑:" + param3;if (param1 == 0) {publisher_event = publisher_event + "截取快照成功..";} else {publisher_event = publisher_event + "截取快照失敗..";}break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:publisher_event = "RTSP服務URL: " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:publisher_event ="服務器不支持RTSP推送, 推送的RTSP URL: " + param3;break;}String str = "當前回調狀態:" + publisher_event;Log.i(TAG, str);Message message = new Message();message.what = PUBLISHER_EVENT_MSG;message.obj = publisher_event;handler.sendMessage(message);}}3. 檢測/設置軟硬編碼:
/*** Set Video H.264 HW Encoder, if support HW encoder, it will return 0(設置H.264硬編碼)* * @param kbps: the kbps of different resolution(25 fps).* * @return {0} if successful*/public native int SetSmartPublisherVideoHWEncoder(long handle, int kbps);/*** Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(設置H.265硬編碼)** @param kbps: the kbps of different resolution(25 fps).** @return {0} if successful*/public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);4. 設置文字水印、PNG圖片水印:
/*** Set Text water-mark(設置文字水印)* * @param fontSize: it should be "MEDIUM", "SMALL", "BIG"* * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".* * @param xPading, yPading: the distance of the original picture.* * <pre> The interface is only used for setting font water-mark when publishing stream. </pre> * * @return {0} if successful*/public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);/*** Set Text water-mark font file name(設置文字水印字體路徑)** @param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf** @return {0} if successful*/public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);/*** Set picture water-mark(設置png圖片水印)* * @param picPath: the picture working path, e.g: /sdcard/logo.png* * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".* * @param picWidth, picHeight: picture width & height* * @param xPading, yPading: the distance of the original picture.* * <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre> * * @return {0} if successful*/public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);5. 軟編碼可變碼率設置(如攝像頭采集編碼,在設備性能允許的情況下,可以考慮軟編碼可變碼率):
/*** Set software encode vbr mode(軟編碼可變碼率).** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** is_enable_vbr: if 0: NOT enable vbr mode, 1: enable vbr** video_quality: vbr video quality, range with (1,50), default 23** vbr_max_kbitrate: vbr max encode bit-rate(kbps)** @return {0} if successful*/public native int SmartPublisherSetSwVBRMode(long handle, int is_enable_vbr, int video_quality, int vbr_max_kbitrate);6. 幀率、軟編碼碼率、關鍵幀間隔,編碼速度、軟編碼profile等基礎設定:
/*** Set gop interval(設置I幀間隔)** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** gopInterval: encode I frame interval, the value always > 0** @return {0} if successful*/public native int SmartPublisherSetGopInterval(long handle, int gopInterval);/*** Set software encode video bit-rate(設置視頻軟編碼bit-rate)** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** avgBitRate: average encode bit-rate(kbps)* * maxBitRate: max encode bit-rate(kbps)** @return {0} if successful*/public native int SmartPublisherSetSWVideoBitRate(long handle, int avgBitRate, int maxBitRate);/*** Set fps(設置幀率)** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** fps: the fps of video, range with (1,25).** @return {0} if successful*/public native int SmartPublisherSetFPS(long handle, int fps);/*** Set software video encoder profile(設置視頻編碼profile).** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** profile: the software video encoder profile, range with (1,3).* * 1: baseline profile* 2: main profile* 3: high profile** @return {0} if successful*/public native int SmartPublisherSetSWVideoEncoderProfile(long handle, int profile);/*** Set software video encoder speed(設置視頻軟編碼編碼速度)* * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>* * @param speed: range with(1, 6), the default speed is 6. * * if with 1, CPU is lowest.* if with 6, CPU is highest.* * @return {0} if successful*/public native int SmartPublisherSetSWVideoEncoderSpeed(long handle, int speed);7. 設置audio編碼類型,如AAC編碼、Speex編碼,并設置如AAC編碼碼率等參數:
/*** Set audio encoder type(設置音頻編碼類型)* * @param type: if with 1:AAC, if with 2: SPEEX* * @return {0} if successful*/public native int SmartPublisherSetAudioCodecType(long handle, int type);/*** Set audio encoder bit-rate(設置音頻編碼碼率), 當前只對AAC編碼有效** @param kbit_rate: 碼率(單位是kbps), 如果是0的話將使用默認碼率, 必須大于等于0** @return {0} if successful*/public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);/*** Set speex encoder quality(設置speex編碼質量)* * @param quality: range with (0, 10), default value is 8* * @return {0} if successful*/public native int SmartPublisherSetSpeexEncoderQuality(long handle, int quality);8. 音頻處理,如自動增益控制、噪音抑制:
/*** Set Audio Noise Suppression(設置音頻噪音抑制)* * @param isNS: if with 1:suppress, if with 0: does not suppress* * @return {0} if successful*/public native int SmartPublisherSetNoiseSuppression(long handle, int isNS);/*** Set Audio AGC(設置音頻自動增益控制)* * @param isAGC: if with 1:AGC, if with 0: does not AGC* * @return {0} if successful*/public native int SmartPublisherSetAGC(long handle, int isAGC);9. 音頻混音接口設置:
/*** 設置混音,目前支持兩路音頻混音** @param is_mix: 1混音, 0不混音, 默認不混音** @return {0} if successful*/public native int SmartPublisherSetAudioMix(long handle, int is_mix);10. 實時靜音設置:
/*** Set mute or not during publish stream(設置實時靜音)* * @param isMute: if with 1:mute, if with 0: does not mute* * @return {0} if successful*/public native int SmartPublisherSetMute(long handle, int isMute);11. 音量調整:
/*** 設置輸入音量, 這個接口一般不建議調用, 在一些特殊情況下可能會用, 一般不建議放大音量** @param index: 一般是0和1, 如果沒有混音的只用0, 有混音的話, 0,1分別設置音量** @param volume: 音量,默認是1.0,范圍是[0.0, 5.0], 設置成0靜音, 1音量不變** @return {0} if successful*/public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);12. 前置攝像頭鏡像:
/*** Set mirror(設置前置攝像頭鏡像)* * @param isMirror: if with 1:mirror mode, if with 0: normal mode* * Please note when with "mirror mode", the publisher and player with the same echo direction* * @return {0} if successful*/public native int SmartPublisherSetMirror(long handle, int isMirror);13. 錄像相關配置,如只錄制音頻或視頻,單個文件最大size,錄像存放目錄,開始/暫停/停止錄像:
/*** 音頻錄制開關, 目的是為了更細粒度的去控制錄像, 一般不需要調用這個接口, 這個接口使用場景比如同時推送音視頻,但只想錄制視頻,可以調用這個接口關閉音頻錄制** @param is_recoder: 0: do not recorder; 1: recorder; sdk默認是1** @return {0} if successful*/public native int SmartPublisherSetRecorderAudio(long handle, int is_recoder);/*** 視頻錄制開關, 目的是為了更細粒度的去控制錄像, 一般不需要調用這個接口, 這個接口使用場景比如同時推送音視頻,但只想錄制音頻,可以調用這個接口關閉視頻錄制** @param is_recoder: 0: do not recorder; 1: recorder; sdk默認是1** @return {0} if successful*/public native int SmartPublisherSetRecorderVideo(long handle, int is_recoder);/*** Create file directory(創建錄像存放目錄)* * @param path, E.g: /sdcard/daniulive/rec* * <pre> The interface is only used for recording the stream data to local side. </pre> * * @return {0} if successful*/public native int SmartPublisherCreateFileDirectory(String path);/*** Set recorder directory(設置錄像存放目錄)* * @param path: the directory of recorder file.* * <pre> NOTE: make sure the path should be existed, or else the setting failed. </pre>* * @return {0} if successful*/public native int SmartPublisherSetRecorderDirectory(long handle, String path);/*** Set the size of every recorded file(設置單個錄像文件大小,如超過最大文件大小,自動切換到下個文件錄制)* * @param size: (MB), (5M~500M), if not in this range, set default size with 200MB.* * @return {0} if successful*/public native int SmartPublisherSetRecorderFileMaxSize(long handle, int size);/*** Start recorder(開始錄像)** @return {0} if successful*/public native int SmartPublisherStartRecorder(long handle);/*** Pause recorder(暫停/恢復錄像)** is_pause: 1表示暫停, 0表示恢復錄像, 輸入其他值將調用失敗** @return {0} if successful*/public native int SmartPublisherPauseRecorder(long handle, int is_pause);/*** Stop recorder(停止錄像)** @return {0} if successful*/public native int SmartPublisherStopRecorder(long handle);14. 實時快照:
/*** Set if needs to save image during publishing stream(設置是否啟用快照)** @param is_save_image: if with 1, it will save current image via the interface of SmartPlayerSaveImage(), if with 0: does not it** @return {0} if successful*/public native int SmartPublisherSaveImageFlag(long handle, int is_save_image);/*** Save current image during publishing stream(實時快照)** @param imageName: image name, which including fully path, "/sdcard/daniuliveimage/daniu.png", etc.** @return {0} if successful*/public native int SmartPublisherSaveCurImage(long handle, String imageName);15. 設置推送的RTMP URL:
/*** Set rtmp publish stream url(設置推送的RTMP url)** @param url: rtmp publish url.** @return {0} if successful*/public native int SmartPublisherSetURL(long handle, String url);16.?Android攝像頭前后camera通過OnPreviewFrame()回調的數據接口:
@Overridepublic void onPreviewFrame(byte[] data, Camera camera) {frameCount++;if (frameCount % 3000 == 0) {Log.i("OnPre", "gc+");System.gc();Log.i("OnPre", "gc-");}if (data == null) {Parameters params = camera.getParameters();Size size = params.getPreviewSize();int bufferSize = (((size.width | 0x1f) + 1) * size.height * ImageFormat.getBitsPerPixel(params.getPreviewFormat())) / 8;camera.addCallbackBuffer(new byte[bufferSize]);} else {if (isRTSPPublisherRunning || isPushingRtmp || isRecording || isPushingRtsp) {libPublisher.SmartPublisherOnCaptureVideoData(publisherHandle, data, data.length, currentCameraType, currentOrigentation);}camera.addCallbackBuffer(data);}}?對應接口定義:
/*** Set live video data(no encoded data).** @param cameraType: CAMERA_FACING_BACK with 0, CAMERA_FACING_FRONT with 1*?* @param curOrg:* PORTRAIT = 1;????//豎屏* LANDSCAPE = 2;????//橫屏 home鍵在右邊的情況* LANDSCAPE_LEFT_HOME_KEY = 3; //橫屏 home鍵在左邊的情況** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoData(long handle, byte[] data, int len, int cameraType, int curOrg);17. 部分定制設備,只支持YV12的數據:
????/*** YV12數據接口** @param data: YV12 data** @param width: 圖像寬** @param height: 圖像高** @param y_stride: ?y面步長** @param v_stride: v面步長** @param u_stride: u面步長** rotation_degree: 順時針旋轉, 必須是0, 90, 180, 270** @return {0} if successful*/public native int SmartPublisherOnYV12Data(long handle, byte[] data, int width, int height, int y_stride, ?int v_stride, int u_stride, int rotation_degree);18. 支持NV21數據接口:
nv21數據接口,除了用于常規的camera數據接入外,部分定制攝像頭出來的數據發生翻轉,這個接口也支持。
????/*** NV21數據接口** @param data: nv21 data** @param len: data length** @param width: 圖像寬** @param height: 圖像高** @param y_stride: ?y面步長** @param uv_stride: ?uv面步長** rotation_degree: 順時針旋轉, 必須是0, 90, 180, 270** @return {0} if successful*/public native int SmartPublisherOnNV21Data(long handle, byte[] data, int len, int width, int height, int y_stride, ?int uv_stride, int rotation_degree);/*** NV21數據接口** @param data: nv21 data** @param len: data length** @param width: 圖像寬** @param height: 圖像高** @param y_stride: ?y面步長** @param uv_stride: ?uv面步長** rotation_degree: 順時針旋轉, 必須是0, 90, 180, 270** @param ?is_vertical_flip: 是否垂直翻轉, 0不翻轉, 1翻轉** @param ?is_horizontal_flip:是否水平翻轉, 0不翻轉, 1翻轉** @return {0} if successful*/public native int SmartPublisherOnNV21DataV2(long handle, byte[] data, int len, int width, int height, int y_stride, ?int uv_stride, int rotation_degree,int is_vertical_flip, int is_horizontal_flip);19. 支持YUV數據接入:
????/*** Set live video data(no encoded data).** @param data: I420 data*?* @param len: I420 data length*?* @param yStride: y stride*?* @param uStride: u stride*?* @param vStride: v stride** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoI420Data(long handle, ?byte[] data, int len, int yStride, int uStride, int vStride);
20. 支持RGBA數據接入(支持裁剪后數據接入,主要用于同屏場景):
????/*** Set live video data(no encoded data).** @param data: RGBA data*?* @param rowStride: stride information*?* @param width: width*?* @param height: height** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGBAData(long handle, ?ByteBuffer data, int rowStride, int width, int height);/*** 投遞裁剪過的RGBA數據** @param data: RGBA data** @param rowStride: stride information** @param width: width** @param height: height** @param clipedLeft: 左; ?clipedTop: 上; clipedwidth: 裁剪后的寬; clipedHeight: 裁剪后的高; 確保傳下去裁剪后的寬、高均為偶數** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ?ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);/*** Set live video data(no encoded data).** @param data: ABGR flip vertical(垂直翻轉) data** @param rowStride: stride information** @param width: width** @param height: height** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoABGRFlipVerticalData(long handle, ?ByteBuffer data, int rowStride, int width, int height);
21. 支持RGB565數據接入(主要用于同屏場景):
????/*** Set live video data(no encoded data).** @param data: RGB565 data** @param row_stride: stride information** @param width: width** @param height: height** @return {0} if successful*/public native int SmartPublisherOnCaptureVideoRGB565Data(long handle,ByteBuffer data, int row_stride, int width, int height);22.?支持camera數據接入(主要用于camera2接口對接):
????
/** ?專門為android.media.Image的android.graphics.ImageFormat.YUV_420_888格式提供的接口** @param ?width: 必須是8的倍數** @param ?height: 必須是8的倍數** @param ?crop_left: 剪切左上角水平坐標, 一般根據android.media.Image.getCropRect() 填充** @param ?crop_top: 剪切左上角垂直坐標, 一般根據android.media.Image.getCropRect() 填充** @param ?crop_width: 必須是8的倍數, 填0將忽略這個參數, 一般根據android.media.Image.getCropRect() 填充** @param ?crop_height: 必須是8的倍數, 填0將忽略這個參數,一般根據android.media.Image.getCropRect() 填充** @param y_plane 對應android.media.Image.Plane[0].getBuffer()** @param y_row_stride 對應android.media.Image.Plane[0].getRowStride()** @param u_plane 對應android.media.Image.Plane[1].getBuffer()** @param v_plane 對應android.media.Image.Plane[2].getBuffer()** @param uv_row_stride 對應android.media.Image.Plane[1].getRowStride()** @param uv_pixel_stride 對應android.media.Image.Plane[1].getPixelStride()** @param ?rotation_degree: 順時針旋轉, 必須是0, 90, 180, 270** @param ?is_vertical_flip: 是否垂直翻轉, 0不翻轉, 1翻轉** @param ?is_horizontal_flip:是否水平翻轉, 0不翻轉, 1翻轉** @param ?scale_width: 縮放寬,必須是8的倍數, 0不縮放** @param ?scale_height: 縮放高, 必須是8的倍數, 0不縮放** @param ?scale_filter_mode: 縮放質量, 范圍必須是[1,3], 傳0使用默認速度** @return {0} if successful*/public native int SmartPublisherOnImageYUV420888(long handle, int width, int height,int crop_left, int crop_top, int crop_width, int crop_height,ByteBuffer y_plane, int y_row_stride,ByteBuffer u_plane, ByteBuffer v_plane, int uv_row_stride, int uv_pixel_stride,int rotation_degree, int is_vertical_flip, int is_horizontal_flip,int scale_width, int scale_height, int scale_filter_mode);
23.?支持PCM數據接入:
?
????/*** 傳遞PCM音頻數據給SDK, 每10ms音頻數據傳入一次*?* ?@param pcmdata: pcm數據, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* ?@param size: pcm數據大小* ?@param sample_rate: 采樣率,當前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推薦44100* ?@param channel: 通道, 當前通道支持單通道(1)和雙通道(2),推薦單通道(1)* ?@param per_channel_sample_number: 這個請傳入的是 sample_rate/100*/public native int SmartPublisherOnPCMData(long handle, ByteBuffer pcmdata, int size, int sample_rate, int channel, int per_channel_sample_number);/*** 傳遞PCM音頻數據給SDK, 每10ms音頻數據傳入一次** ?@param pcmdata: pcm數據, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* ?@param offset: pcmdata的偏移* ?@param size: pcm數據大小* ?@param sample_rate: 采樣率,當前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推薦44100* ?@param channel: 通道, 當前通道支持單通道(1)和雙通道(2),推薦單通道(1)* ?@param per_channel_sample_number: 這個請傳入的是 sample_rate/100*/public native int SmartPublisherOnPCMDataV2(long handle, ByteBuffer pcmdata, int offset, int size, int sample_rate, int channel, int per_channel_sample_number);/*** 傳遞PCM音頻數據給SDK, 每10ms音頻數據傳入一次** ?@param pcm_short_array: pcm數據, short是native endian order* ?@param offset: 數組偏移* ?@param len: 數組項數* ?@param sample_rate: 采樣率,當前只支持{44100, 8000, 16000, 24000, 32000, 48000}, 推薦44100* ?@param channel: 通道, 當前通道支持單通道(1)和雙通道(2),推薦單通道(1)* ?@param per_channel_sample_number: 這個請傳入的是 sample_rate/100*/public native int SmartPublisherOnPCMShortArray(long handle, short[] pcm_short_array, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);
24.?支持遠端PCM數據接入和混音后PCM數據接入(主要用于一對一互動):? ??
/*** Set far end pcm data*?* @param pcmdata : 16bit pcm data* @param sampleRate: audio sample rate* @param channel: auido channel* @param per_channel_sample_number: per channel sample numbers* @param is_low_latency: if with 0, it is not low_latency, if with 1, it is low_latency* @return {0} if successful*/public native int SmartPublisherOnFarEndPCMData(long handle, ?ByteBuffer pcmdata, int sampleRate, int channel, int per_channel_sample_number, int is_low_latency);/*** 傳遞PCM混音音頻數據給SDK, 每10ms音頻數據傳入一次** ?@param stream_index: 當前只能傳1, 傳其他返回錯誤* ?@param pcm_data: pcm數據, 需要使用ByteBuffer.allocateDirect分配, ByteBuffer.isDirect()是true的才行.* ?@param offset: pcmdata的偏移* ?@param size: pcm數據大小* ?@param sample_rate: 采樣率,當前只支持{44100, 8000, 16000, 24000, 32000, 48000}* ?@param channels: 通道, 當前通道支持單通道(1)和雙通道(2)* ?@param per_channel_sample_number: 這個請傳入的是 sample_rate/100*/public native int SmartPublisherOnMixPCMData(long handle, int stream_index, ByteBuffer pcm_data, int offset, int size, int sample_rate, int channels, int per_channel_sample_number);/*** 傳遞PCM混音音頻數據給SDK, 每10ms音頻數據傳入一次** ?@param stream_index: 當前只能傳1, 傳其他返回錯誤* ?@param pcm_short_array: pcm數據, short是native endian order* ?@param offset: 數組偏移* ?@param len: 數組項數* ?@param sample_rate: 采樣率,當前只支持{44100, 8000, 16000, 24000, 32000, 48000}* ?@param channels: 通道, 當前通道支持單通道(1)和雙通道(2)* ?@param per_channel_sample_number: 這個請傳入的是 sample_rate/100*/public native int SmartPublisherOnMixPCMShortArray(long handle, int stream_index, short[] pcm_short_array, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);25. H264擴展SEI消息:
/*** 設置發送隊列大小,為保證實時性,默認大小為3, 必須設置一個大于0的數** @param max_size: 隊列最大長度** @param reserve: 保留字段** NOTE: 1. 如果數據超過隊列大小,將丟掉隊頭數據; 2. 這個接口請在 StartPublisher 之前調用** @return {0} if successful*/public native int SmartPublisherSetPostUserDataQueueMaxSize(long handle, int max_size, int reserve);/*** 清空用戶數據隊列, 有些情況可能會用到,比如發送隊列里面有4條消息再等待發送,又想把最新的消息快速發出去, 可以先清除掉正在排隊消息, 再調用PostUserXXX** @return {0} if successful*/public native int SmartPublisherClearPostUserDataQueue(long handle);/*** 發送二進制數據** NOTE:* 1.目前數據大小限制在256個字節以內,太大可能會影響視頻傳輸,如果有特殊需求,需要增大限制,請聯系我們* 2. 如果積累的數據超過了設置的隊列大小,之前的隊頭數據將被丟棄* 3. 必須再調用StartPublisher之后再發送數據** @param data: 二進制數據** @param size: 數據大小** @param reserve: 保留字段** @return {0} if successful*/public native int SmartPublisherPostUserData(long handle, byte[] data, int size, int reserve);/*** 發送utf8字符串** NOTE:* 1. 字符串長度不能超過256, 太大可能會影響視頻傳輸,如果有特殊需求,需要增大限制,請聯系我們* 2. 如果積累的數據超過了設置的隊列大小,之前的隊頭數據將被丟棄* 3. 必須再調用StartPublisher之后再發送數據** @param utf8_str: utf8字符串** @param reserve: 保留字段** @return {0} if successful*/public native int SmartPublisherPostUserUTF8StringData(long handle, String utf8_str, int reserve);26. 關閉推送實例:
/*** 關閉推送實例,結束時必須調用close接口釋放資源** @return {0} if successful*/public native int SmartPublisherClose(long handle);總結
以上是生活随笔為你收集整理的如何在Android实现录屏直播的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 【Python】分享几个用Python给
- 下一篇: 【机器学习】微软出品!FLAML:一款可