tombstone墓碑生成机制

Android中程序在运行时会遇到各种各样的问题,相应的就会产生各种异常信号,比如常见的异常信号 Singal 11:Segmentation fault表示无效的地址进行了操作,比如内存越界、空指针调用等。
Android中在进程(主要指native进程)崩溃时会生成墓碑文件,这些文件中记录了崩溃时的调用堆栈、日志信息、寄存器二进制数据等等,用以帮助开发者已经崩溃问题。
墓碑文件默认保存在**/data/tombstones/**目录中,以tombstone_xxx(xxx表示编号)方式命名。墓碑文件数量有上限,达到上限时会删除最旧的墓碑文件,可以通过配置属性 tombstoned.max_tombstone_count来修改默认的墓碑文件数量。

本文源码基于Android12版本。

墓碑环境初始化

bionic为程序初始化墓碑生成环境

bionic是android提供的符合POSIX接口的标准C库,其中提供了Linker(动态连接器)。动态链接器的作用是在运行动态链接的可执行文件时,动态链接器负责加载程序到内存中,并解析对符号的引用。
bionic在Linker中初始化墓碑生成环境,下面的汇编代码中执行了__linker_init这个符号(函数)

//bionic/linker/arch/arm64/begin.S#include ENTRY(_start)// Force unwinds to end in this function..cfi_undefined x30mov x0, spbl __linker_init/* linker init returns the _entry address in the main image */br x0END(_start)

__linker_init这个函数定义在/bionic/linker/linker_main.cpp中,先后执行__linker_init、__linker_init_post_relocation、linker_main。在linker_main函数中,调用linker_debuggerd_init,初始化墓碑生成环境。另外,在linker_main函数中可以看到很多比较重要的初始化函数,比如__system_properties_init。

//bionic/linker/linker_main.cppextern "C" ElfW(Addr) __linker_init(void* raw_args) {// Initialize TLS early so system calls and errno work. // 省略return __linker_init_post_relocation(args, tmp_linker_so);}static ElfW(Addr) __attribute__((noinline))__linker_init_post_relocation(KernelArgumentBlock& args, soinfo& tmp_linker_so) {// 省略// 执行linker_mainElfW(Addr) start_address = linker_main(args, exe_to_load);if (g_is_ldd) _exit(EXIT_SUCCESS);INFO("[ Jumping to _start (%p)... ]", reinterpret_cast<void*>(start_address));// Return the address that the calling assembly stub should jump to.return start_address;}static ElfW(Addr) linker_main(KernelArgumentBlock& args, const char* exe_to_load) {ProtectedDataGuard guard;#if TIMINGstruct timeval t0, t1;gettimeofday(&t0, 0);#endif// Sanitize the environment.__libc_init_AT_SECURE(args.envp);// Initialize system properties__system_properties_init(); // may use 'environ'// Initialize platform properties.platform_properties_init();// 这里!!!// Register the debuggerd signal handler.linker_debuggerd_init();// 省略return entry;}

linker_debuggerd_init函数定义在/bionic/linker/linker_debuggerd_android.cpp中,该函数调用了libdebuggerd_handler_core库(/system/core/debuggerd)的debuggerd_init函数。

//bionic/linker/linker_debuggerd_android.cppvoid linker_debuggerd_init() {// There may be a version mismatch between the bootstrap linker and the crash_dump in the APEX,// so don't pass in any process info from the bootstrap linker.debuggerd_callbacks_t callbacks = {#if defined(__ANDROID_APEX__).get_process_info = get_process_info,#endif.post_dump = notify_gdb_of_libraries,};// 这里debuggerd_init(&callbacks);}
debuggerd模块为Signal安装处理的Handler

debuggerd_init函数中,会为各个异常信号Signal注册用来处理信号的Handler。这样当程序发生异常时,就会调用注册好的Handler。

//system/core/debuggerd/handler/debuggerd_handler.cppvoid debuggerd_init(debuggerd_callbacks_t* callbacks) {// 省略// linux sigaction的标准用法struct sigaction action;memset(&action, 0, sizeof(action));sigfillset(&action.sa_mask);// debuggerd_signal_handler就是用来处理异常信号的Handleraction.sa_sigaction = debuggerd_signal_handler;action.sa_flags = SA_RESTART | SA_SIGINFO;// Use the alternate signal stack if available so we can catch stack overflows.action.sa_flags |= SA_ONSTACK;#define SA_EXPOSE_TAGBITS 0x00000800// Request that the kernel set tag bits in the fault address. This is necessary for diagnosing MTE// faults.action.sa_flags |= SA_EXPOSE_TAGBITS;// 为各个异常信号注册Handlerdebuggerd_register_handlers(&action);}

debuggerd_register_handlers函数在头文件中实现(这种形式叫内联函数)。可以通过ro.debuggabledebug.debuggerd.disable属性来控制注册过程。

//system/core/debuggerd/include/debuggerd/handler.hstatic void __attribute__((__unused__)) debuggerd_register_handlers(struct sigaction* action) {char value[PROP_VALUE_MAX] = "";bool enabled =!(__system_property_get("ro.debuggable", value) > 0 && !strcmp(value, "1") &&__system_property_get("debug.debuggerd.disable", value) > 0 && !strcmp(value, "1"));if (enabled) {// 针对不同异常注册sigaction(SIGABRT, action, nullptr);sigaction(SIGBUS, action, nullptr);sigaction(SIGFPE, action, nullptr);sigaction(SIGILL, action, nullptr);sigaction(SIGSEGV, action, nullptr);sigaction(SIGSTKFLT, action, nullptr);sigaction(SIGSYS, action, nullptr);sigaction(SIGTRAP, action, nullptr);}sigaction(BIONIC_SIGNAL_DEBUGGER, action, nullptr);}

到此墓碑环境注册完成,在这个流程中可以选择通过ro.debuggable或debug.debuggerd.disable来关闭墓碑。

墓碑生成流程

完成了上述墓碑环境初始化后,当程序运行发生异常,比如内存越界触发了SIGSEGV就会调用debuggerd_signal_handler这个函数(

//system/core/debuggerd/handler/debuggerd_handler.cpp// Handler that does crash dumping by forking and doing the processing in the child.// Do this by ptracing the relevant thread, and then execing debuggerd to do the actual dump.static void debuggerd_signal_handler(int signal_number, siginfo_t* info, void* context) {// 省略// clone一个子进程出来(在clone出来的进程中处理墓碑生成)// Essentially pthread_create without CLONE_FILES, so we still work during file descriptor// exhaustion.pid_t child_pid =clone(debuggerd_dispatch_pseudothread, pseudothread_stack,CLONE_THREAD | CLONE_SIGHAND | CLONE_VM | CLONE_CHILD_SETTID | CLONE_CHILD_CLEARTID,&thread_info, nullptr, nullptr, &thread_info.pseudothread_tid);if (child_pid == -1) {fatal_errno("failed to spawn debuggerd dispatch thread");}// Wait for the child to start...futex_wait(&thread_info.pseudothread_tid, -1);// and then wait for it to terminate.futex_wait(&thread_info.pseudothread_tid, child_pid);// 后面是一些收尾处理// Restore PR_SET_DUMPABLE to its original value.if (prctl(PR_SET_DUMPABLE, orig_dumpable) != 0) {fatal_errno("failed to restore dumpable");}// Restore PR_SET_PTRACER to its original value.if (restore_orig_ptracer && prctl(PR_SET_PTRACER, 0) != 0) {fatal_errno("failed to restore traceable");}if (info->si_signo == BIONIC_SIGNAL_DEBUGGER) {// If the signal is fatal, don't unlock the mutex to prevent other crashing threads from// starting to dump right before our death.pthread_mutex_unlock(&crash_mutex);} else {// Resend the signal, so that either the debugger or the parent's waitpid sees it.resend_signal(info);}}

上面的函数中,clone了一个子进程来处理了墓碑生成流程。clone出来的子进程会执行debuggerd_dispatch_pseudothread函数。

static int debuggerd_dispatch_pseudothread(void* arg) {// 省略// 创建pipe管理(因为后面还要fork一个进程来执行crash_dump64这个bin程序)// pipe用来与之后fork的进程通信用unique_fd input_read, input_write;unique_fd output_read, output_write;if (!Pipe(&input_read, &input_write) != 0 || !Pipe(&output_read, &output_write)) {fatal_errno("failed to create pipe");}// fork一个子进程// Don't use fork(2) to avoid calling pthread_atfork handlers.pid_t crash_dump_pid = __fork();if (crash_dump_pid == -1) {async_safe_format_log(ANDROID_LOG_FATAL, "libc","failed to fork in debuggerd signal handler: %s", strerror(errno));} else if (crash_dump_pid == 0) {// 省略// 子进程执行 "/apex/com.android.runtime/bin/crash_dump64 这个程序// crash_dump64程序是墓碑文件真正的生成者execle(CRASH_DUMP_PATH, CRASH_DUMP_NAME, main_tid, pseudothread_tid, debuggerd_dump_type, nullptr, nullptr);async_safe_format_log(ANDROID_LOG_FATAL, "libc", "failed to exec crash_dump helper: %s",strerror(errno));return 1;}// 省略}

在debuggerd_dispatch_pseudothread中主要做了两个事件,一个是创建Pipe用来与子进程通信。一个是fork了一个子进程,让子进程执行crash_dump64这个二进制程序。crash_dump64这个二进制程序中会真正的生成墓碑文件。
crash_dump64的实现在/system/core/debuggerd/crash_dump.cpp

int main(int argc, char** argv) {// 省略// 判断debug.debuggerd.wait_for_debugger,是否等待gdb// Defer the message until later, for readability.bool wait_for_debugger = android::base::GetBoolProperty("debug.debuggerd.wait_for_debugger",android::base::GetBoolProperty("debug.debuggerd.wait_for_gdb", false));if (siginfo.si_signo == BIONIC_SIGNAL_DEBUGGER) {wait_for_debugger = false;}// 连接tombstoned守护进程,通过tombstoned得到墓碑文件的FD(g_output_fd){ATRACE_NAME("tombstoned_connect");LOG(INFO) << "obtaining output fd from tombstoned, type: " << dump_type;g_tombstoned_connected = tombstoned_connect(g_target_thread, &g_tombstoned_socket, &g_output_fd,&g_proto_fd, dump_type);}// 使用unwindstack生成函数调用堆栈// TODO: Use seccomp to lock ourselves down.unwindstack::UnwinderFromPid unwinder(256, vm_pid, unwindstack::Regs::CurrentArch());if (!unwinder.Init()) {LOG(FATAL) << "Failed to init unwinder object.";}// 生成墓碑文件中的内容std::string amfd_data;if (backtrace) {ATRACE_NAME("dump_backtrace");dump_backtrace(std::move(g_output_fd), &unwinder, thread_info, g_target_thread);} else {{ATRACE_NAME("fdsan table dump");populate_fdsan_table(&open_files, unwinder.GetProcessMemory(), process_info.fdsan_table_address);}{ATRACE_NAME("engrave_tombstone");// 这里,生成墓碑engrave_tombstone(std::move(g_output_fd), std::move(g_proto_fd), &unwinder, thread_info,g_target_thread, process_info, &open_files, &amfd_data);}}// return 0;}

crash_dump64会连接tombstoned这个进程,通过tombstoned取得将要输出的墓碑文件的FD(因为墓碑文件有数量限制、达到上限时要删除旧的墓碑文件,所以专门用tombstoned这个守护进程管理)。然后使用unwindstack库生成函数堆栈,并调用
engrave_tombstone函数生成墓碑。

在engrave_tombstone函数中,我们会看到比较熟悉的墓碑文件中的文本内容。比如“***”这种字符。另外只有在ro.debuggable开启的状态下,才会调用dump_logs在墓碑文件中输出Log日志。

//system/core/debuggerd/libdebuggerd/tombstone.cppvoid engrave_tombstone(unique_fd output_fd, unique_fd proto_fd, unwindstack::Unwinder* unwinder, const std::map& threads, pid_t target_thread, const ProcessInfo& process_info, OpenFilesList* open_files, std::string* amfd_data) {// Don't copy log messages to tombstone unless this is a development device.Tombstone tombstone;engrave_tombstone_proto(&tombstone, unwinder, threads, target_thread, process_info, open_files);if (proto_fd != -1) {if (!tombstone.SerializeToFileDescriptor(proto_fd.get())) {async_safe_format_log(ANDROID_LOG_ERROR, LOG_TAG, "failed to write proto tombstone: %s",strerror(errno));}}log_t log;log.current_tid = target_thread;log.crashed_tid = target_thread;log.tfd = output_fd.get();log.amfd_data = amfd_data;bool translate_proto = GetBoolProperty("debug.debuggerd.translate_proto_to_text", true);if (translate_proto) {tombstone_proto_to_text(tombstone, [&log](const std::string& line, bool should_log) {_LOG(&log, should_log " />second, process_info, true);if (want_logs) {dump_logs(&log, it->second.pid, 50);}for (auto& [tid, thread_info] : threads) {if (tid == target_thread) {continue;}dump_thread(&log, unwinder, thread_info, process_info, false);}if (open_files) {_LOG(&log, logtype::OPEN_FILES, "\nopen files:\n");dump_open_files_list(&log, *open_files, "");}if (want_logs) {dump_logs(&log, it->second.pid, 0);}}}

总结

墓碑初始化及生成流程中,可以通过属性控制是否注册墓碑、是否生成墓碑,以及墓碑文件的数量等功能。同时,也可以根据业务需求,在墓碑中加入自定义内容,比如给墓碑文件的名字追加特殊的时间戳、追加一些自定义日志到墓碑中等等。