Android ART VM的文件格式－OAT（四）

一，与oat相关的文件后缀

１）.oat，OAT是由dex2oat产生的，本质上也是属于elf文件。

２）.odex，在Dalvik中，odex表示被优化后的dex文件；ART虚拟机中，它实际上是oat文件。

oat文件除了遵循elf文件规范，又根据虚拟机的需求进行了扩展－－最大的区别增加了两个重要的字段 oat data section 和oat exec section，其中data section保存的是原dex文件中的字节码数据，exec section是dex经过dex2oat编译后生成的机器码的存储区域。可以从data section中通过一定的对应关系可以迅速找到某个class/function在exec section中机器码。

加载oat文件的函数是dlopen。

art/runtime/oat_file.cc

bool DlOpenOatFile::Dlopen(const std::string& elf_filename,
                           uint8_t* oat_file_begin,
                           std::string* error_msg) {
#ifdef __APPLE__
#else
  {
    UniqueCPtr absolute_path(realpath(elf_filename.c_str(), nullptr));
    if (absolute_path == nullptr) {
      *error_msg = StringPrintf("Failed to find absolute path for '%s'", elf_filename.c_str());
      return false;
    }
#ifdef __ANDROID__
    android_dlextinfo extinfo;
    extinfo.flags = ANDROID_DLEXT_FORCE_LOAD |                  // Force-load, don't reuse handle
                                                                //   (open oat files multiple
                                                                //    times).
                    ANDROID_DLEXT_FORCE_FIXED_VADDR;            // Take a non-zero vaddr as absolute
                                                                //   (non-pic boot image).
    if (oat_file_begin != nullptr) {                            //
      extinfo.flags |= ANDROID_DLEXT_LOAD_AT_FIXED_ADDRESS;     // Use the requested addr if
      extinfo.reserved_addr = oat_file_begin;                   // vaddr = 0.
    }                                                           //   (pic boot image).
    dlopen_handle_ = android_dlopen_ext(absolute_path.get(), RTLD_NOW, &extinfo);
#else
#endif
  }
  if (dlopen_handle_ == nullptr) {
    *error_msg = StringPrintf("Failed to dlopen '%s': %s", elf_filename.c_str(), dlerror());
    return false;
  }
  return true;
#endif
}

OAT文件的内部结构，可以拜读老罗的博客。

二，OAT的编译时机

１）Rom构建的时候，

/build/core/Dex_preopt.mk

include $(BUILD_SYSTEM)/dex_preopt_libart.mk

# Define dexpreopt-one-file based on current default runtime.
# $(1): the input .jar or .apk file
# $(2): the output .odex file
define dexpreopt-one-file
$(call dex2oat-one-file,$(1),$(2))
endef

在系统构建时会执行dex2oat编译。art程序的预优化脚本是dex_preopt_libart.mk。

dex2oat是在编译的那个阶段执行的优化操作呢？实际是在需要执行优化的所有模块（包括jar,apk），在其产生dex文件的地方，开始执行优化的，具体就是调用dexpreopt-one-file脚本，为dex2oat的执行提供上下文环境。

dex2oat程序支持的选项参数：

-j :指定执行编译操作所需的线程数量。

--dex-file= 输入参数: 指定需要被编译的.dex, .jar,或apk文件（内含.dex），如：--dex-file=/system/framework/core.jar

--oat-file= 输出参数：以文件名的形式指定oat的输出目标。如：--oat-file=/system/framework/boot.oat

--oat-symbols= 输出参数：指定带有完全符号表的OAT输出目标，如：--oat-file=/symbols/system/framework/boot.oat

......

２）第三方应用安装的时候，执行dex2oat，在点击安装apk文件时，是怎样关联到Android系统的安装服务的？

应用安装的起点是startActivity的调用，响应安装请求的是packageinstallerActivity，然后调用InstallAppProcess进入下一步的操作，InstallAppProcess一方面显示安装进度，一方面调用系统服务packageManagerservice执行具体的安装过程。

PMS在安装apk的过程中，会跟installd（init.rc中启动的daemon进程）这个系统服务通信。PMS中的变量mInstaller是PMS跟installd沟通的桥梁，Installer基于socket来建立跟installd的通信链接。

如果PMS判断要安装的apk需要dex2oat编译，会调用Installer的函数dexopt：

framework/base/services/core/java/com/android/server/pm/Installer.java

    public void dexopt(String apkPath, int uid, String instructionSet, int dexoptNeeded,
            int dexFlags, String compilerFilter, String volumeUuid, String sharedLibraries)
            throws InstallerException {
        assertValidInstructionSet(instructionSet);
        mInstaller.dexopt(apkPath, uid, instructionSet, dexoptNeeded, dexFlags,
                compilerFilter, volumeUuid, sharedLibraries);
    }

这个函数中mInstaller指的是InstallerConnection实例，也就是连接installd的一个通道。

framework/base/core/java/com/android/internal/os/InstallerConnection.java

    public void dexopt(String apkPath, int uid, String pkgName, String instructionSet,
            int dexoptNeeded, String outputPath, int dexFlags, String compilerFilter,
            String volumeUuid, String sharedLibraries) throws InstallerException {
        execute("dexopt",
                apkPath,
                uid,
                pkgName,
                instructionSet,
                dexoptNeeded,
                outputPath,
                dexFlags,
                compilerFilter,
                volumeUuid,
                sharedLibraries);
    }

先构造传递到installd的参数，然后通过execute执行，真正往installd发送命令是通过transact函数。

public synchronized String transact(String cmd) {
//如果连接还没建立，要先去执行连接
        if (!connect()) {
            Slog.e(TAG, "connection failed");
            return "-1";
        }
//向installd写入命令
        if (!writeCommand(cmd)) {
            /*
             * If installd died and restarted in the background (unlikely but
             * possible) we'll fail on the next write (this one). Try to
             * reconnect and write the command one more time before giving up.
             */
　　　　　　　//写入失败，再次尝试连接、写入数据
　　　　　　　Slog.e(TAG, "write command failed? reconnect!");
            if (!connect() || !writeCommand(cmd)) {
                return "-1";
            }
        }
//读取installd的执行结果
        final int replyLength = readReply();
        if (replyLength > 0) {
            String s = new String(buf, 0, replyLength);
            return s;
        } else {
            return "-1";
        }
    }

这样命令就从PMS发送到installd了，installd作为一个daemon进程，启动后会在一个循环中等待连接，然后接收命令，处理命令。

frameworks/native/cmds/installd/installd.cpp

static int execute(int s, char cmd[BUFFER_MAX])
{
    char reply[REPLY_MAX];
    char *arg[TOKEN_MAX+1];
    unsigned i;
    unsigned n = 0;
    unsigned short count;
    int ret = -1;
        /* default reply is "" */
    reply[0] = 0;

        /* n is number of args (not counting arg[0]) */
    arg[0] = cmd;
    while (*cmd) {
??? //循环处理所有参数，多个参数之间是以空格做分隔符的，跟发送端的格式一致
?????????if (isspace(*cmd)) {
            *cmd++ = 0;
            n++;
            arg[n] = cmd;
            if (n == TOKEN_MAX) {
                ALOGE("too many arguments\n");
                goto done;
            }
        }
        if (*cmd) {
          cmd++;
        }
    }
//cmds是一个数组，定义了命令跟处理函数之间的映射
    for (i = 0; i < sizeof(cmds) / sizeof(cmds[0]); i++) {
        if (!strcmp(cmds[i].name,arg[0])) {
            if (n != cmds[i].numargs) {
                ALOGE("%s requires %d arguments (%d given)\n",
                     cmds[i].name, cmds[i].numargs, n);
            } else {
                ret = cmds[i].func(arg + 1, reply);
            }
            goto done;
        }
    }
    ALOGE("unsupported command '%s'\n", arg[0]);

done:
    if (reply[0]) {
        n = snprintf(cmd, BUFFER_MAX, "%d %s", ret, reply);
    } else {
        n = snprintf(cmd, BUFFER_MAX, "%d", ret);
    }
    if (n > BUFFER_MAX) n = BUFFER_MAX;
    count = n;

    // ALOGI("reply: '%s'\n", cmd);
    if (writex(s, &count, sizeof(count))) return -1;
    if (writex(s, cmd, count)) return -1;
    return 0;
}

这个数组是commond跟执行函数的映射关系：

struct cmdinfo cmds[] = {
    { "ping",                 0, do_ping },

    { "create_app_data",      7, do_create_app_data },
    { "restorecon_app_data",  6, do_restorecon_app_data },
    { "migrate_app_data",     4, do_migrate_app_data },
    { "clear_app_data",       5, do_clear_app_data },
    { "destroy_app_data",     5, do_destroy_app_data },
    { "move_complete_app",    7, do_move_complete_app },
    { "get_app_size",         6, do_get_app_size },
    { "get_app_data_inode",   4, do_get_app_data_inode },

    { "create_user_data",     4, do_create_user_data },
    { "destroy_user_data",    3, do_destroy_user_data },

    { "dexopt",              10, do_dexopt },
    { "markbootcomplete",     1, do_mark_boot_complete },
    { "rmdex",                2, do_rm_dex },
    { "freecache",            2, do_free_cache },
    { "linklib",              4, do_linklib },
    { "idmap",                3, do_idmap },
    { "createoatdir",         2, do_create_oat_dir },
    { "rmpackagedir",         1, do_rm_package_dir },
    { "clear_app_profiles",   1, do_clear_app_profiles },
    { "destroy_app_profiles", 1, do_destroy_app_profiles },
    { "linkfile",             3, do_link_file },
    { "move_ab",              3, do_move_ab },
    { "merge_profiles",       2, do_merge_profiles },
    { "dump_profiles",        3, do_dump_profiles },
    { "delete_odex",          3, do_delete_odex },
};

可以看到dexopt对应是do_dexopt函数。

static int do_dexopt(char **arg, char reply[REPLY_MAX])
{
    const char* args[DEXOPT_PARAM_COUNT];
    for (size_t i = 0; i < DEXOPT_PARAM_COUNT; ++i) {
        CHECK(arg[i] != nullptr);
        args[i] = arg[i];
    }

    int dexopt_flags = atoi(arg[6]);
    DexoptFn dexopt_fn;
    if ((dexopt_flags & DEXOPT_OTA) != 0) {
        dexopt_fn = do_ota_dexopt;
    } else {
        dexopt_fn = do_regular_dexopt;
    }
    return dexopt_fn(args, reply);
}

以普通应用安装，是执行do_regular_dexopt，最终的实现是调用了Commonds.cpp中的dexopt函数：

frameworks/native/cmds/installd/Commonds.cpp

int dexopt(const char* apk_path, uid_t uid, const char* pkgname, const char* instruction_set,
           int dexopt_needed, const char* oat_dir, int dexopt_flags, const char* compiler_filter,
           const char* volume_uuid ATTRIBUTE_UNUSED, const char* shared_libraries)
{
    bool is_public = ((dexopt_flags & DEXOPT_PUBLIC) != 0);
    bool vm_safe_mode = (dexopt_flags & DEXOPT_SAFEMODE) != 0;
    bool debuggable = (dexopt_flags & DEXOPT_DEBUGGABLE) != 0;
    bool boot_complete = (dexopt_flags & DEXOPT_BOOTCOMPLETE) != 0;
    bool profile_guided = (dexopt_flags & DEXOPT_PROFILE_GUIDED) != 0;

    char out_path[PKG_PATH_MAX];
//这里会创建dalvik-cache文件
?????if (!create_oat_out_path(apk_path, instruction_set, oat_dir, out_path)) {
        return false;
    }

    const char *input_file;
    char in_odex_path[PKG_PATH_MAX];
    switch (dexopt_needed) {
        case DEXOPT_DEX2OAT_NEEDED:
            input_file = apk_path;
            break;

        case DEXOPT_PATCHOAT_NEEDED:
            if (!calculate_odex_file_path(in_odex_path, apk_path, instruction_set)) {
                return -1;
            }
            input_file = in_odex_path;
            break;

        case DEXOPT_SELF_PATCHOAT_NEEDED:
            input_file = out_path;
            break;

        default:
            ALOGE("Invalid dexopt needed: %d\n", dexopt_needed);
            return 72;
    }

    struct stat input_stat;
    memset(&input_stat, 0, sizeof(input_stat));
    stat(input_file, &input_stat);
//打开输入文件，也即是apk文件
    base::unique_fd input_fd(open(input_file, O_RDONLY, 0));
    if (input_fd.get() < 0) {
        ALOGE("installd cannot open '%s' for input during dexopt\n", input_file);
        return -1;
    }

    const std::string out_path_str(out_path);
//创建输出结果文件
?????Dex2oatFileWrapper> out_fd(
            open_output_file(out_path, /*recreate*/true, /*permissions*/0644),
            [out_path_str]() { unlink(out_path_str.c_str()); });
    if (out_fd.get() < 0) {
        ALOGE("installd cannot open '%s' for output during dexopt\n", out_path);
        return -1;
    }
    if (!set_permissions_and_ownership(out_fd.get(), is_public, uid, out_path)) {
        return -1;
    }
    ALOGV("DexInv: --- BEGIN '%s' ---\n", input_file);
//启动一个子进程，来真正执行dex2oat的编译，因为dex2oat是一个独立的程序，所以为它的运行创建一个新的进程
    pid_t pid = fork();
    if (pid == 0) {
        /* child -- drop privileges before continuing */
        drop_capabilities(uid);

        SetDex2OatAndPatchOatScheduling(boot_complete);
        if (flock(out_fd.get(), LOCK_EX | LOCK_NB) != 0) {
            ALOGE("flock(%s) failed: %s\n", out_path, strerror(errno));
            _exit(67);
        }

        if (dexopt_needed == DEXOPT_PATCHOAT_NEEDED
            || dexopt_needed == DEXOPT_SELF_PATCHOAT_NEEDED) {
            run_patchoat(input_fd.get(),
                         out_fd.get(),
                         input_file,
                         out_path,
                         pkgname,
                         instruction_set);
        } else if (dexopt_needed == DEXOPT_DEX2OAT_NEEDED) {
            // Pass dex2oat the relative path to the input file.
            const char *input_file_name = get_location_from_path(input_file);
??????? //这个调用是实际执行编译的地方
??????????? run_dex2oat(input_fd.get(),
                        out_fd.get(),
                        image_fd.get(),
                        input_file_name,
                        out_path,
                        swap_fd.get(),
                        instruction_set,
                        compiler_filter,
                        vm_safe_mode,
                        debuggable,
                        boot_complete,
                        reference_profile_fd.get(),
                        shared_libraries);
        } else {
            ALOGE("Invalid dexopt needed: %d\n", dexopt_needed);
            _exit(73);
        }
        _exit(68);   /* only get here on exec failure */
    } else {
//父进程要等着子进程任务完成。
?????????int res = wait_child(pid);
        if (res == 0) {
            ALOGV("DexInv: --- END '%s' (success) ---\n", input_file);
        } else {
            ALOGE("DexInv: --- END '%s' --- status=0x%04x, process failed\n", input_file, res);
            return -1;
        }
    }

    struct utimbuf ut;
    ut.actime = input_stat.st_atime;
    ut.modtime = input_stat.st_mtime;
    utime(out_path, &ut);

    // We've been successful, don't delete output.
    out_fd.SetCleanup(false);
    image_fd.SetCleanup(false);
    reference_profile_fd.SetCleanup(false);

    return 0;
}

到这里，第三方apk的dex2oat的编译就完成了。