我正在编写一个应用程序，它将从远程 TCP 连接实时接收的一系列图片呈现到 ImageView 中。元素。
该流由以 PGM 格式编码并以 9Hz 发送的单帧组成，我认为像这样的非常低的帧速率应该使用背景轻松处理 Service将完全解码的位图发送到我的MainActivity .
这是我的VideoService (我只发布了 run() 方法，因为我认为这是唯一一个感兴趣的方法):
public void run() {
InetAddress serverAddr = null;
try {
serverAddr = InetAddress.getByName(VIDEO_SERVER_ADDR);
} catch (UnknownHostException e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
return;
}
Socket socket = null;
BufferedReader reader = null;
do {
try {
socket = new Socket(serverAddr, VIDEO_SERVER_PORT);
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
boolean frameStart = false;
LinkedList<String> frameList = new LinkedList<>();
while (keepRunning) {
final String message = reader.readLine();
if (!frameStart && message.startsWith("F"))
frameStart = true;
else if (frameStart && message.startsWith("EF")) {
frameStart = false;
final Bitmap bitmap = Bitmap.createBitmap(IR_FRAME_WIDTH, IR_FRAME_HEIGHT, Bitmap.Config.ARGB_8888);
final Canvas canvas = new Canvas(bitmap);
final String[] data = frameList.toArray(new String[frameList.size()]);
canvas.drawBitmap(bitmap, 0, 0, null);
//Log.d(this.getClass().getName(), "IR FRAME COLLECTED");
if ((data.length - 6) == IR_FRAME_HEIGHT) {
float grayScaleRatio = Float.parseFloat(data[2].trim()) / 255.0f;
for (int y = 0; y < IR_FRAME_HEIGHT; y++) {
final String line = data[y + 3];
final String[] points = line.split("\\s+");
if (points.length == IR_FRAME_WIDTH) {
for (int x = 0; x < IR_FRAME_WIDTH; x++) {
final float grayLevel = Float.parseFloat(points[x]) / grayScaleRatio;
Paint paint = new Paint();
paint.setStyle(Paint.Style.FILL);
final int level = (int)grayLevel;
paint.setColor(Color.rgb(level, level, level));
canvas.drawPoint(x, y, paint);
}
} else
Log.d(this.getClass().getName(), "Malformed line");
}
final Intent messageIntent = new Intent();
messageIntent.setAction(VIDEO_BROADCAST_KEY);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
bitmap.recycle();
messageIntent.putExtra(VIDEO_MESSAGE_KEY, stream.toByteArray());
stream.close();
sendBroadcast(messageIntent);
} else
Log.d(this.getClass().getName(), "Malformed data");
frameList.clear();
} else if (frameStart)
frameList.add(message);
}
Thread.sleep(VIDEO_SERVER_RESPAWN);
} catch (Throwable e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
}
} while (keepRunning);
if (socket != null) {
try {
socket.close();
} catch (Throwable e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
}
}
}
message是来自以下文本的一行:F
P2
160 120
1226
193 141 158 152 193 186 171 177 186 160 195 182 ... (160 times)
.
. (120 lines)
.
278 248 253 261 257 284 310 304 304 272 227 208 ... (160 times)
EF
在 MainActivity 我通过这段代码处理这个:
class VideoReceiver extends BroadcastReceiver {
final public Queue<Bitmap> imagesQueue = new LinkedList<>();
@Override
public void onReceive(Context context, Intent intent) {
try {
//Log.d(getClass().getName(), "onReceive() called");
final byte[] data = intent.getByteArrayExtra(VideoService.VIDEO_MESSAGE_KEY);
final Bitmap bitmap = BitmapFactory.decodeByteArray(data,0,data.length);
imagesQueue.add(bitmap);
runOnUiThread(updateVideoTask);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
updateVideoTask任务定义如下: updateVideoTask = new Runnable() {
public void run() {
if (videoReceiver == null) return;
if (!videoReceiver.imagesQueue.isEmpty())
{
final Bitmap image = videoReceiver.imagesQueue.poll();
if (image == null) return;
videoView.setImageBitmap(image);
Log.d(this.getClass().getName(), "Images to spool: " + videoReceiver.imagesQueue.size());
}
}
};
不幸的是，当我运行应用程序时，我注意到帧速率非常低并且延迟非常大。我不能争论发生了什么。
我从 得到的唯一提示日志猫 这些行是:
2019-05-20 16:37:08.817 29566-29580/it.tux.gcs I/art: Background sticky concurrent mark sweep GC freed 88152(3MB) AllocSpace objects, 3(52KB) LOS objects, 22% free, 7MB/10MB, paused 3.937ms total 111.782ms
2019-05-20 16:37:08.832 29566-29587/it.tux.gcs D/skia: Encode PNG Singlethread : 13003 us, width=160, height=120
即使所有这些延迟(140 毫秒)的总和，应用程序也应该维持超过 5Hz 的帧速率，而我得到 0.25Hz 甚至更差。
经过一番调查，我发现移动:
Paint paint = new Paint();
paint.setStyle(Paint.Style.FILL);
在嵌套循环之外阻止 GC 被如此频繁地调用，我在这一行中发现了另一个主要的延迟来源:
final String[] points = line.split("\\s+");
它每次烧掉 2 毫秒，所以我决定选择不那么聪明但速度更快的东西:
final String[] points = line.split(" ");
无论如何它仍然不够..之间的代码:
canvas.drawBitmap(bitmap, 0, 0, null);
和
sendBroadcast(messageIntent);
仍然消耗超过 200ms ......我怎样才能做得比这更好？
我很确定有一种更有效的方法可以从 TCP 服务器收集一系列具有这种大小和速率的帧，并将它们显示在
ImageView 上。 .当然，这可能是软件架构的问题，而不仅仅是代码本身的优化。除了 native 代码之外，我对任何新方法都持开放态度(我不熟悉它)。
更新(2019 年 3 月 11 日):
Activity 方面:
public class MainActivity extends AppCompatActivity implements FrameReadyCallBack {
private Intent videoServiceIntent;
private VideoService videoService;
private boolean bound = false;
private ImageView surfaceView_video = null;
private String videoPort = "5002";
private String videoServerAddr = "192.168.10.107";
private ServiceConnection serviceConnection = null;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
surfaceView_video = findViewById(R.id.surfaceView_video);
serviceConnection = new ServiceConnection() {
@Override
public void onServiceConnected(ComponentName className, IBinder service) {
VideoService.VideoServiceBinder binder = (VideoService.VideoServiceBinder) service;
videoService = binder.getService();
bound = true;
videoService.registerCallBack(MainActivity.this); // register
}
@Override
public void onServiceDisconnected(ComponentName arg0) {
bound = false;
}
};
startVideoService();
}
@Override
public void frameReady(byte[] image_data) {
//TODO: create image and update surfaceView_video
}
public void startVideoService()
{
videoServiceIntent = new Intent(this, VideoService.class);
videoServiceIntent.putExtra(VideoService.LOCAL_PORT_KEY, videoPort);
videoServiceIntent.putExtra(VideoService.LOCAL_VIDEOSERVER_ADDR_KEY, videoServerAddr);
startService(videoServiceIntent);
}
@Override
protected void onStart() {
super.onStart();
bindService();
}
@Override
protected void onStop() {
super.onStop();
unbindService();
}
private void bindService() {
bindService(videoServiceIntent, serviceConnection, Context.BIND_AUTO_CREATE);
}
private void unbindService(){
if (bound) {
videoService.registerCallBack(null); // unregister
unbindService(serviceConnection);
bound = false;
}
}
}
服务端:
public class VideoService extends Service {
public static final String LOCAL_PORT_KEY = "video_port";
public static final String LOCAL_VIDEOSERVER_ADDR_KEY = "video_server_addr";
private static final int DEFAULT_VIDEO_PORT = 5002;
private static final int VIDEO_SERVER_RESPAWN = 2000;
private volatile FrameReadyCallBack frameReadyCallBack = null;
private VideoReceiver videoReceiver = null;
private IBinder videoServiceBinder = new VideoServiceBinder();
@Nullable
@Override
public IBinder onBind(Intent intent) {
return videoServiceBinder ;
}
@Override
public boolean onUnbind(Intent intent) {
videoReceiver.kill();
return super.onUnbind(intent);
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
final int localVideoPort = intent.getIntExtra(LOCAL_PORT_KEY, DEFAULT_VIDEO_PORT);
final String videoServerAddr = intent.getStringExtra(LOCAL_VIDEOSERVER_ADDR_KEY);
videoReceiver = new VideoReceiver(videoServerAddr, localVideoPort);
videoReceiver.start();
return Service.START_NOT_STICKY;
}
public void registerCallBack(FrameReadyCallBack frameReadyCallBack) {
this.frameReadyCallBack = frameReadyCallBack;
}
public class VideoServiceBinder extends Binder {
public VideoService getService() {
return VideoService.this;
}
}
private class VideoReceiver extends Thread {
private boolean keepRunning = true;
private int VIDEO_SERVER_PORT;
private String VIDEO_SERVER_ADDR;
private int bad_frames;
private int frames;
private int link_respawn;
private FrameDecodingStatus status;
public VideoReceiver(String addr, int listen_port) {
VIDEO_SERVER_PORT = listen_port;
VIDEO_SERVER_ADDR = addr;
}
public void run() {
InetAddress serverAddr;
link_respawn = 0;
try {
serverAddr = InetAddress.getByName(VIDEO_SERVER_ADDR);
} catch (UnknownHostException e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
return;
}
Socket socket = null;
DataInputStream stream;
do {
bad_frames = 0;
frames = 0;
status = FrameDecodingStatus.Idle;
try {
socket = new Socket(serverAddr, VIDEO_SERVER_PORT);
stream = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
final byte[] _data = new byte[PACKET_SIZE];
final byte[] _image_data = new byte[IMAGE_SIZE];
int _data_index = 0;
while (keepRunning) {
if (stream.read(_data, 0, _data.length) == 0)
continue;
for (byte _byte : _data) {
if (status == FrameDecodingStatus.Idle) {
//Wait SoM
} else if (status == FrameDecodingStatus.Data) {
//Collect data
} else {
frameReadyCallBack.frameReady(_image_data);
status = FrameDecodingStatus.Idle;
}
}
}
link_respawn++;
Thread.sleep(VIDEO_SERVER_RESPAWN);
Log.d(getClass().getName(), "Link respawn: " + link_respawn);
} catch (Throwable e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
}
} while (keepRunning);
if (socket != null) {
try {
socket.close();
} catch (Throwable e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
}
}
}
public void kill() {
keepRunning = false;
}
}
}
最佳答案
首先，您通过 BroadcastReceiver 提交带有新图像更改的结果。由于某些原因。您可以显着提高整体速度，但删除此逻辑。并将通信替换为 Service通过bound features .
// Bind to LocalService
Intent intent = new Intent(this, LocalService.class);
bindService(intent, connection, Context.BIND_AUTO_CREATE);
然后接收连接。
/** Defines callbacks for service binding, passed to bindService() */
private ServiceConnection connection = new ServiceConnection() {
@Override
public void onServiceConnected(ComponentName className,
IBinder service) {
// We've bound to LocalService, cast the IBinder and get LocalService instance
LocalBinder binder = (LocalBinder) service;
mService = binder.getService();
mBound = true;
}
@Override
public void onServiceDisconnected(ComponentName arg0) {
mBound = false;
}
};
然后使用
Service要订阅的活页夹实例 Activity并在 Service 中使用回调发布新的数据字节。
关于android - 位图创建优化，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/56223708/