在Android中使用Protocol Buffers
生活随笔
收集整理的這篇文章主要介紹了
在Android中使用Protocol Buffers
小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.
網(wǎng)絡(luò)性能優(yōu)化的終極手法就是不通過網(wǎng)絡(luò)傳輸，但這常常是不可能的。但我們還是可以通過對網(wǎng)絡(luò)傳輸?shù)臄?shù)據(jù)本身做優(yōu)化，來獲得更好的性能，性能就應(yīng)該從每一個可能的地方榨取。這里來看一下 Protocol Buffers 。
Protocol Buffers 是一個序列化結(jié)構(gòu)數(shù)據(jù)的靈活、高效且自動化的機(jī)制——類似于XML，但更小，更快，更簡單。定義一次結(jié)構(gòu)化數(shù)據(jù)的方式，然后就可以使用專門生成的代碼簡單地寫入，或用不同的語言從大量的數(shù)據(jù)流讀出結(jié)構(gòu)化數(shù)據(jù)。甚至可以更新數(shù)據(jù)結(jié)構(gòu)而不破壞已部署的基于 老 格式編譯的程序。我們看一下要如何將 Protocol Buffers 用到我們的Android項(xiàng)目中。
總覽
先來看一下 Protocol Buffers 項(xiàng)目已經(jīng)為我們提供了什么，我們在使用 Protocol Buffers 時需要做什么的整體流程。如下圖：
Protocol Buffers Architecture
在使用 Protocol Buffers 時，我們需要以特殊的方式定義我們的結(jié)構(gòu)化數(shù)據(jù)，保存為 .proto 消息定義文件。 Protocol Buffers 項(xiàng)目為我們提供了編譯器，可以將 .proto 文件編譯為Java文件以用于我們的Java 或 Android應(yīng)用項(xiàng)目。這個編譯器在我們的PC機(jī)上編譯并運(yùn)行。產(chǎn)生的Java文件是依賴于 Protocol Buffers 的Java庫的，比如這些文件實(shí)現(xiàn)了庫的借口等。我們將生成的這些Java文件和 Protocol Buffers 的Java庫引入我們的Android應(yīng)用項(xiàng)目，就可以方便地以 Protocol Buffers 的二進(jìn)制格式操作結(jié)構(gòu)化數(shù)據(jù)了。
每次手動執(zhí)行 Protocol Buffers 編譯器將 .proto 文件轉(zhuǎn)換為Java文件顯然有點(diǎn)太麻煩了。 Protocol Buffers 項(xiàng)目的開發(fā)者顯然也想到了這一點(diǎn)，因而他們還為我們提供了一個Android Studio gradle插件 protobuf-gradle-plugin ，以便于在我們項(xiàng)目的編譯期間自動地執(zhí)行 Protocol Buffers 編譯器。
我們可以為 protobuf-gradle-plugin 指定本地 Protocol Buffers 編譯器的路徑讓它使用本地的編譯執(zhí)行編譯，也可以使用 Protocol Buffers 項(xiàng)目提供的另外一個工具，在編譯時動態(tài)地下載并執(zhí)行編譯過程。
后面我們詳細(xì)地來看這個過程。
下載編譯Protocol編譯器
我們可以在如下位置：
https://github.com/google/protobuf/releases下載打包好的protobuf，也可以直接clone protobuf的代碼，自己手動編譯編譯器。這里我們從GitHub上clone代碼并手動編譯編譯器：
$ git clone https://github.com/google/protobuf.git 正克隆到 'protobuf'... remote: Counting objects: 38993, done. remote: Compressing objects: 100% (17/17), done. remote: Total 38993 (delta 4), reused 0 (delta 0), pack-reused 38974 接收對象中: 100% (38993/38993), 36.14 MiB | 239.00 KiB/s, 完成. 處理 delta 中: 100% (26220/26220), 完成. 檢查連接... 完成。下載代碼之后，進(jìn)入protobuf目錄并執(zhí)行 autogen.sh ：
$ cd protobuf $ ./autogen.sh Google Mock not present. Fetching gmock-1.7.0 from the web...% Total % Received % Xferd Average Speed Time Time Time CurrentDload Upload Total Spent Left Speed 100 129 0 129 0 0 111 0 --:--:-- 0:00:01 --:--:-- 112 100 362k 100 362k 0 0 67764 0 0:00:05 0:00:05 --:--:-- 92816% Total % Received % Xferd Average Speed Time Time Time CurrentDload Upload Total Spent Left Speed 100 129 0 129 0 0 112 0 --:--:-- 0:00:01 --:--:-- 112 100 618k 100 618k 0 0 56321 0 0:00:11 0:00:11 --:--:-- 115k + autoreconf -f -i -Wall,no-obsolete libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'. ......這個腳本主要用于下載測試用的gmock-1.7.0，并生成用于編譯配置的 configure 等文件。可以通過如下命令了解我們可以對protobuf的編譯做哪些配置，以及默認(rèn)配置的信息：
$ ./configure --help `configure' configures Protocol Buffers 3.1.0 to adapt to many kinds of systems.Usage: ./configure [OPTION]... [VAR=VALUE]...To assign environment variables (e.g., CC, CFLAGS...), specify them as VAR=VALUE. See below for descriptions of some of the useful variables.Defaults for the options are specified in brackets.Configuration:-h, --help display this help and exit--help=short display options specific to this package--help=recursive display the short help of all the included packages-V, --version display version information and exit-q, --quiet, --silent do not print `checking ...' messages--cache-file=FILE cache test results in FILE [disabled]-C, --config-cache alias for `--cache-file=config.cache'-n, --no-create do not create output files--srcdir=DIR find the sources in DIR [configure dir or `..']Installation directories:--prefix=PREFIX install architecture-independent files in PREFIX[/usr/local]--exec-prefix=EPREFIX install architecture-dependent files in EPREFIX[PREFIX]By default, `make install' will install all the files in `/usr/local/bin', `/usr/local/lib' etc. You can specify an installation prefix other than `/usr/local' using `--prefix', for instance `--prefix=$HOME'.For better control, use the options below.Fine tuning of the installation directories:--bindir=DIR user executables [EPREFIX/bin] ......執(zhí)行configure對編譯進(jìn)行配置：
$ ./configure checking whether to enable maintainer-specific portions of Makefiles... yes checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking target system type... x86_64-pc-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p ......這樣就生成了makefile文件，編譯并安裝：
$ make $ sudo make install這個過程在編譯并安裝 Protocol Buffers 編譯器之外，還會為host編譯用于支持在C++中使用 Protocol Buffers 的庫。（編譯生成的二進(jìn)制文加在 protobuf/src/.libs 下。）
安裝之后執(zhí)行如下命令以確認(rèn)已經(jīng)裝好：
$ protoc --version libprotoc 3.1.0在執(zhí)行protoc時通過給它加上 --help 參數(shù)可以了解到這個工具更多的用法。
$ protoc --help Usage: protoc [OPTION] PROTO_FILES Parse PROTO_FILES and generate output based on the options given:-IPATH, --proto_path=PATH Specify the directory in which to search forimports. May be specified multiple times;directories will be searched in order. If notgiven, the current working directory is used.--version Show version info and exit.-h, --help Show this text and exit.--encode=MESSAGE_TYPE Read a text-format message of the given typefrom standard input and write it in binaryto standard output. The message type mustbe defined in PROTO_FILES or their imports.--decode=MESSAGE_TYPE Read a binary message of the given type fromstandard input and write it in text formatto standard output. The message type mustbe defined in PROTO_FILES or their imports.--decode_raw Read an arbitrary protocol message fromstandard input and write the raw tag/valuepairs in text format to standard output. NoPROTO_FILES should be given when using thisflag.-oFILE, Writes a FileDescriptorSet (a protocol buffer,--descriptor_set_out=FILE defined in descriptor.proto) containing all ofthe input files to FILE.--include_imports When using --descriptor_set_out, also includeall dependencies of the input files in theset, so that the set is self-contained.--include_source_info When using --descriptor_set_out, do not stripSourceCodeInfo from the FileDescriptorProto.This results in vastly larger descriptors thatinclude information about the originallocation of each decl in the source file aswell as surrounding comments.--dependency_out=FILE Write a dependency output file in the formatexpected by make. This writes the transitiveset of input file paths to FILE--error_format=FORMAT Set the format in which to print errors.FORMAT may be 'gcc' (the default) or 'msvs'(Microsoft Visual Studio format).--print_free_field_numbers Print the free field numbers of the messagesdefined in the given proto files. Groups sharethe same field number space with the parent message. Extension ranges are counted as occupied fields numbers.--plugin=EXECUTABLE Specifies a plugin executable to use.Normally, protoc searches the PATH forplugins, but you may specify additionalexecutables not in the path using this flag.Additionally, EXECUTABLE may be of the formNAME=PATH, in which case the given plugin nameis mapped to the given executable even ifthe executable's own name differs.--cpp_out=OUT_DIR Generate C++ header and source.--csharp_out=OUT_DIR Generate C# source file.--java_out=OUT_DIR Generate Java source file.--javanano_out=OUT_DIR Generate Java Nano source file.--js_out=OUT_DIR Generate JavaScript source.--objc_out=OUT_DIR Generate Objective C header and source.--php_out=OUT_DIR Generate PHP source file.--python_out=OUT_DIR Generate Python source file.--ruby_out=OUT_DIR Generate Ruby source file.創(chuàng)建 .proto 文件
.proto 文件中的定義很簡單：為每個想要序列化的數(shù)據(jù)結(jié)構(gòu)添加一個 消息(message) ，然后為消息中的每個字段指定一個名字和類型以及一個tag數(shù)字。如官方提供的一個例子addressbook.proto：
package tutorial;option java_package = "com.example.tutorial"; option java_outer_classname = "AddressBookProtos";message Person {required string name = 1;required int32 id = 2;optional string email = 3;enum PhoneType {MOBILE = 0;HOME = 1;WORK = 2;}message PhoneNumber {required string number = 1;optional PhoneType type = 2 [default = HOME];}repeated PhoneNumber phone = 4; }message AddressBook {repeated Person person = 1; }可以參考 在Java中使用Protocol Buffers 一文了解更多關(guān)于創(chuàng)建 .proto 文件的基礎(chǔ)知識。
編譯 .proto 文件
可以通過如下命令編譯 .proto 文件：
protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto-I，--java_out 分別用于指定源目錄 (放置應(yīng)用程序源代碼的地方 —— 如果沒有提供則使用當(dāng)前目錄)，目的目錄 (希望放置生成的代碼的位置；通常與$SRC_DIR相同)，最后的參數(shù)為 .proto 文件的路徑。protoc會按照標(biāo)準(zhǔn)Java風(fēng)格，生成Java類及目錄結(jié)構(gòu)。如對于上面的例子，會生成 com/example/tutorial/ 目錄結(jié)構(gòu)，及 AddressBookProtos.java 文件。
在Android項(xiàng)目中使用 Protocol Buffers
我們將 由 .proto 文件生成的Java文件復(fù)制到我們的Android項(xiàng)目中：
在我們app的build.gradle中添加對 protobuf-java 的依賴，就像依賴其它那些Java庫一樣：
dependencies {compile fileTree(dir: 'libs', include: ['*.jar'])testCompile 'junit:junit:4.12'compile 'com.android.support:appcompat-v7:23.4.0'compile 'com.google.protobuf:protobuf-java:3.0.0' }添加訪問Protocol Buffers的類的類。這里我們添加兩個類，AddPerson用于構(gòu)造Person對象：
package com.netease.volleydemo;import com.example.tutorial.AddressBookProtos.Person;public class AddPerson {static Person createPerson(String personName) {Person.Builder person = Person.newBuilder();int id = 13958235;person.setId(id);String name = personName;person.setName(name);String email = "zhangsan@gmail.com";person.setEmail(email);Person.PhoneNumber.Builder phoneNumber = Person.PhoneNumber.newBuilder();phoneNumber.setType(Person.PhoneType.HOME);phoneNumber.setNumber("0157-23443276");person.addPhone(phoneNumber.build());phoneNumber = Person.PhoneNumber.newBuilder();phoneNumber.setType(Person.PhoneType.MOBILE);phoneNumber.setNumber("136183667387");person.addPhone(phoneNumber.build());return person.build();} }AddressBookProtobuf類則用于編碼/解碼AddressBook對象：
package com.netease.volleydemo;import com.example.tutorial.AddressBookProtos;import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.InputStream;public class AddressBookProtobuf {public static byte[] encodeTest(String[] names) {AddressBookProtos.AddressBook.Builder addressBook = AddressBookProtos.AddressBook.newBuilder();for(int i = 0; i < names.length; ++ i) {addressBook.addPerson(AddPerson.createPerson(names[i]));}AddressBookProtos.AddressBook book = addressBook.build();ByteArrayOutputStream baos = new ByteArrayOutputStream();try {book.writeTo(baos);} catch (IOException e) {}return baos.toByteArray();}public static byte[] encodeTest(String[] names, int times) {for (int i = 0; i < times - 1; ++ i) {encodeTest(names);}return encodeTest(names);}public static AddressBookProtos.AddressBook decodeTest(InputStream is) {AddressBookProtos.AddressBook addressBook = null;try {addressBook = AddressBookProtos.AddressBook.parseFrom(is);} catch (IOException e) {e.printStackTrace();}return addressBook;}public static AddressBookProtos.AddressBook decodeTest(InputStream is, int times) {AddressBookProtos.AddressBook addressBook = null;for (int i = 0; i < times; ++ i) {addressBook = decodeTest(is);}return addressBook;} }使用protobuf-gradle-plugin
每次單獨(dú)執(zhí)行protoc編譯 .proto 文件總是太麻煩，通過protobuf-gradle-plugin可以在編譯我們的app時自動地編譯 .proto 文件，這樣就大大降低了我們在Android項(xiàng)目中使用 Protocol Buffers 的難度。
首先我們需要將 .proto 文件添加進(jìn)我們的項(xiàng)目中，如：
然后修改 app/build.gradle 對protobuf gradle插件做配置：
添加protobuf塊，對protobuf-gradle-plugin的執(zhí)行做配置：
protobuf {protoc {path = '/usr/local/bin/protoc'}generateProtoTasks {all().each { task ->task.builtins {remove java}task.builtins {java { }// Add cpp output without any option.// DO NOT omit the braces if you want this builtin to be added.cpp { }}}} }protoc塊用于配置Protocol Buffers編譯器，這里我們指定用我們之前手動編譯的編譯器。
task.builtins的塊必不可少，這個塊用于指定我們要為那些編程語言生成代碼，這里我們?yōu)镃++和Java生成代碼。缺少這個塊的話，在編譯時會報(bào)出如下的錯誤：
提示說沒有指定輸出目錄的路徑。
這是由于 protobuf-gradle-plugin 執(zhí)行的protobuf編譯器命令的參數(shù)是在protobuf-gradle-plugin/src/main/groovy/com/google/protobuf/gradle/GenerateProtoTask.groovy中構(gòu)造的：
可以看到，輸出目錄是由builtins構(gòu)造的。
對前面的protobuf塊做一點(diǎn)點(diǎn)修改，我們甚至來編譯protobuf編譯器都不需要了。修改如下：
protobuf {protoc {artifact = 'com.google.protobuf:protoc:3.0.0'}generateProtoTasks {all().each { task ->task.builtins {remove java}task.builtins {java { }cpp { }}}} }關(guān)于 .proto 文件的編寫方法，Protocol Buffers API等更多內(nèi)容，可以參考 Protobuf開發(fā)者指南、在Java中使用Protocol Buffers及其它相關(guān)官方文檔。
Protobuf 與 JSON 對比測試
說了半天，Protobuf的表現(xiàn)究竟如何呢？這里我們就對比一下我們最常用到的JSON格式與Protobuf的表現(xiàn)。測試基于在開發(fā)者中一向有著良好口碑的fastjson進(jìn)行。
測試用的數(shù)據(jù)結(jié)構(gòu)如我們前面看到的AddressBook。我們通過構(gòu)造包含不同個數(shù)Person的AddressBook數(shù)據(jù)，并對這些數(shù)據(jù)執(zhí)行多次編碼解碼操作，來測試Protobuf 與 JSON的表現(xiàn)。Protobuf的編碼/解碼測試代碼如前面看到的AddressBookProtobuf。JSON的測試代碼則如下面這樣：
package com.netease.volleydemo;import com.alibaba.fastjson.JSON;import java.util.ArrayList; import java.util.List;public class AddressBookJson {private enum PhoneType {MOBILE,HOME,WORK}private static final class Phone {private String number;private PhoneType type;public Phone() {}public void setNumber(String number) {this.number = number;}public String getNumber() {return number;}public void setType(PhoneType phoneType) {this.type = phoneType;}public PhoneType getType() {return type;}}private static final class Person {private String name;private int id;private String email;private List<Phone> phones;public Person() {phones = new ArrayList<>();}public void setName(String name) {this.name = name;}public String getName() {return name;}public void setId(int id) {this.id = id;}public int getId() {return id;}public void setEmail(String email) {this.email = email;}public String getEmail() {return email;}public void addPhone(Phone phone) {phones.add(phone);}public List<Phone> getPhones() {return phones;}}private static final class AddressBook {private List<Person> persons;public AddressBook() {persons = new ArrayList<>();}public void addPerson(Person person) {persons.add(person);}public List<Person> getPersons() {return persons;}}public static String encodeTest(String[] names) {AddressBook addressBook = new AddressBook();for (int i = 0; i < names.length; ++ i) {Person person = new Person();person.setName(names[i]);person.setEmail("zhangsan@gmail.com");person.setId(13958235);Phone phone = new Phone();phone.setNumber("0157-23443276");phone.setType(PhoneType.HOME);person.addPhone(phone);phone = new Phone();phone.setNumber("136183667387");phone.setType(PhoneType.MOBILE);person.addPhone(phone);addressBook.addPerson(person);}String jsonString = JSON.toJSONString(addressBook);return jsonString;}public static String encodeTest(String[] names, int times) {for (int i = 0; i < times - 1; ++ i) {encodeTest(names);}return encodeTest(names);}public static AddressBook decodeTest(String jsonStr, int times) {AddressBook addressBook = null;for (int i = 0; i < times; ++ i) {addressBook = JSON.parseObject(jsonStr, AddressBook.class);}return addressBook;} }通過如下的這段代碼來執(zhí)行測試：
private class ProtobufTestTask extends AsyncTask<Void, Void, Void> {private static final int BUFFER_LEN = 8192;private void doEncodeTest(String[] names, int times) {long startTime = System.nanoTime();AddressBookProtobuf.encodeTest(names, times);long protobufTime = System.nanoTime();protobufTime = protobufTime - startTime;startTime = System.nanoTime();AddressBookJson.encodeTest(names, times);long jsonTime = System.nanoTime();jsonTime = jsonTime - startTime;Log.i(TAG, String.format("%-20s%-20s%-20s%-20s", "ProtobufTime", String.valueOf(protobufTime),"JsonTime", String.valueOf(jsonTime)));}private void doEncodeTest10(int times) {doEncodeTest(TestUtils.sTestNames10, times);}private void doEncodeTest50(int times) {doEncodeTest(TestUtils.sTestNames50, times);}private void doEncodeTest100(int times) {doEncodeTest(TestUtils.sTestNames100, times);}private void doEncodeTest(int times) {doEncodeTest10(times);doEncodeTest50(times);doEncodeTest100(times);}private void compress(InputStream is, OutputStream os)throws Exception {GZIPOutputStream gos = new GZIPOutputStream(os);int count;byte data[] = new byte[BUFFER_LEN];while ((count = is.read(data, 0, BUFFER_LEN)) != -1) {gos.write(data, 0, count);}gos.finish();gos.close();}private void doDecodeTest(String[] names, int times) {byte[] protobufBytes = AddressBookProtobuf.encodeTest(names);ByteArrayInputStream bais = new ByteArrayInputStream(protobufBytes);ByteArrayOutputStream baos = new ByteArrayOutputStream();try {compress(bais, baos);} catch (Exception e) {e.printStackTrace();}Log.i(TAG, String.format("%-20s%-20s%-20s%-20s", "Protobuf Length", String.valueOf(protobufBytes.length),"Protobuf(GZIP) Length", String.valueOf(baos.toByteArray().length)));bais = new ByteArrayInputStream(protobufBytes);long startTime = System.nanoTime();AddressBookProtobuf.decodeTest(bais, times);long protobufTime = System.nanoTime();protobufTime = protobufTime - startTime;String jsonStr = AddressBookJson.encodeTest(names);ByteArrayInputStream jsonBais = new ByteArrayInputStream(jsonStr.getBytes());ByteArrayOutputStream jsonBaos = new ByteArrayOutputStream();try {compress(jsonBais, jsonBaos);} catch (Exception e) {e.printStackTrace();}Log.i(TAG, String.format("%-20s%-20s%-20s%-20s", "Json Length", String.valueOf(jsonStr.getBytes().length),"Json(GZIP) Length", String.valueOf(jsonBaos.toByteArray().length)));startTime = System.nanoTime();AddressBookJson.decodeTest(jsonStr, times);long jsonTime = System.nanoTime();jsonTime = jsonTime - startTime;Log.i(TAG, String.format("%-20s%-20s%-20s%-20s", "ProtobufTime", String.valueOf(protobufTime),"JsonTime", String.valueOf(jsonTime)));}private void doDecodeTest10(int times) {doDecodeTest(TestUtils.sTestNames10, times);}private void doDecodeTest50(int times) {doDecodeTest(TestUtils.sTestNames50, times);}private void doDecodeTest100(int times) {doDecodeTest(TestUtils.sTestNames100, times);}private void doDecodeTest(int times) {doDecodeTest10(times);doDecodeTest50(times);doDecodeTest100(times);}@Overrideprotected Void doInBackground(Void... params) {TestUtils.initTest();doEncodeTest(5000);doDecodeTest(5000);return null;}@Overrideprotected void onPostExecute(Void aVoid) {super.onPostExecute(aVoid);}}這里我們執(zhí)行3組編碼測試及3組解碼測試。對于編碼測試，第一組的單個數(shù)據(jù)中包含10個Person，第二組的包含50個，第三組的包含100個，然后對每個數(shù)據(jù)分別執(zhí)行5000次的編碼操作。
對于解碼測試，三組中單個數(shù)據(jù)同樣包含10個Person、50個及100個，然后對每個數(shù)據(jù)分別執(zhí)行5000次的解碼碼操作。
在Galaxy Nexus的Android 4.4.4 CM平臺上執(zhí)行上述測試，最終得到如下結(jié)果：
編碼后數(shù)據(jù)長度對比 (Bytes)
| 10 | 860 | 291 | 1703 | 344 |
| 50 | 4300 | 984 | 8463 | 1047 |
| 100 | 8600 | 1840 | 16913 | 1913 |
相同的數(shù)據(jù)，經(jīng)過Protobuf編碼的數(shù)據(jù)長度，大概只有JSON編碼的數(shù)據(jù)長度的一半。但對編碼后的數(shù)據(jù)再進(jìn)行壓縮，兩者則差別比較小。
編碼性能對比 (S)
| 10 | 4.687 | 6.558 |
| 50 | 23.728 | 41.315 |
| 100 | 45.604 | 81.667 |
編碼性能最少提高了 28.5%，最多則提高了44.2%。Protobuf在編碼性能上，相對于JSON還是有較大幅度的提升的。
解碼性能對比 (S)
| 10 | 0.226 | 8.839 |
| 50 | 0.291 | 43.869 |
| 100 | 0.220 | 85.444 |
解碼性能方面，Protobuf相對于JSON，則更是有驚人的提升。Protobuf的解碼時間幾乎不隨著數(shù)據(jù)長度的增長而有太大的增長，而JSON則隨著數(shù)據(jù)長度的增加，解碼所需要的時間也越來越長。
Done。
總結(jié)
以上是生活随笔為你收集整理的在Android中使用Protocol Buffers的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 在Java中使用Protocol Buf
- 下一篇： 在Android中使用FlatBuffe