How Android works, Part 2

In this article, I'll talk about some of the ideas on which high-level parts of Android are built, about several of its predecessors and about the basic mechanisms of security.

Articles series:

How Android works, part 1
How Android works, Part 2
How Android works, part 3

Speaking about Unix and Linux-roots Android, you need to remember about other projects of operating systems, the influence of which can be traced in Android, although they are not his direct ancestors.
I already mentioned about BeOS, in the legacy from which Android went to Binder.
Plan 9 from Bell Labs
Plan 9 is a descendant of Unix, a logical extension, development of his ideas and bringing them to perfection. Plan 9 was developed in Bell Labs by the same team that created Unix and C - people like Ken Thompson, Rob Pike, Dennis Ritchie, Brian Kernighan, Tom Duff, Doug McIlroy, Bjarne Stroustrup, Bruce Ellis and others worked on it.
In Plan 9, the interaction of processes between themselves and the kernel of the system is realized not through numerous system calls and IPC mechanisms, but through virtual text files and file systems (the development of the Unix principle "everything is a file"). In this case, each group of processes "sees" the file system in its own way (namespaces), which allows you to run different parts of the system in different environments.
For example, to get the position of the mouse cursor, the applications read the text file / dev / mouse. The rio window system provides each application with its own version of this file, in which only the events pertaining to the application window are visible, and coordinates are used locally relative to the window. Rio itself reads the events of the "real" mouse through the same file / dev / mouse - in the form in which it sees it. If it is started directly, this file is provided by the kernel and really describes the movements of this mouse, but it can be transparently launched as an application under another rio copy, without any special support from her.
Plan 9 fully supports access to remote file systems (using its own 9P protocol, in addition, FTP and SFTP are supported), which allows programs to transparently access remote files, interfaces and resources. This "native" network transparency turns Plan 9 into a distributed operating system - the user can physically be behind one computer running rio, run applications on several others, use files stored on the file server and perform calculations on the CPU server - all this is completely transparent and without special support from each part of the system.
Due to the beautifully designed architecture Plan 9 is much easier and less than Unix - in fact, the core of Plan 9 is even several times smaller than the known micronuclear Mach.
<blockquote> Despite the technical superiority and the availability of a compatibility layer with Unix, Plan 9 is not widely used. Nevertheless, many of the ideas and technologies from Plan 9 have become widespread and have been implemented in other systems. The most famous of these is the UTF-8 encoding, which was developed in Plan 9 to provide full support for Unicode while maintaining backward compatibility with ASCII - has become the accepted standard.
The most ideas and technologies from Plan 9 are implemented in Linux:

file system / proc (procfs)
system callclone (similar to rfork from Plan 9)
support for mount namespaces
Support for file systems implemented in the user space (filesystem in userspace, FUSE)
9P protocol support

Much of this is used, including, in Android. In addition, Android has implemented a mechanism of intent, similar to the plumber from Plan 9; about him I'll tell in the next article.
For Plan 9, you can learn more at ( a saved copy in Wayback Machine ), or its mirror
Plan 9 was continued as an Inferno project, also developed at Bell Labs. To such properties of Plan 9 as simplicity and distribution, Inferno adds portability. Programs for Inferno are written in high-level language Limbo and are executed - using just-in-time compilation - built into the core of Inferno virtual machine.
Inferno is so portable that it can be launched

on processors of different architectures: ARM, x86, IBM PowerPC, Sun SPARC, 6SGI MIPS and HP PA-RISC,
as an independent operating system or as a program under Plan 9, Unix, Windows 95 and Windows NT.

In this case, applications that run inside Inferno are given the same environment.
Inferno received even less distribution and fame than Plan 9. On the other hand, Inferno in many ways anticipated Android, the most popular operating system in the world.
Danger Research Inc. was co-founded by Andy Rubin (Andy Rubin) in 1999, four years before co-founded by Android Inc. in the year 2004.
In 2002, Danger released their smartphone Danger Hiptop. Many of the developers of Danger subsequently worked on Android, so it's no wonder that its operating system was much like Android. For example, it implemented:

"Always running" applications written in Java,
full-fledged web browser,
web applications,
email client,
cloud synchronization,
store third-party applications.

More information about Danger can be found in the article by Chris DeSalvo, one of the developers, called The future that everyone forgot .
Although the use of high-level languages ​​for serious development is no longer surprising, from popular operating systems only Android has a "native" language - high-level Java (on the other hand, you can recall the web with its JavaScript, .NET for Windows and relatively high-level - but completely compiled into native code and not using garbage collection - Swift).
Despite the seeming shortcomings ("Java combines the beauty of C ++ syntax with the speed of python execution"), Java has many advantages.
First, Java is the most popular (with a large margin) programming language. Java has a huge ecosystem of libraries and development tools (including assembly systems and IDEs). About Java written a lot of articles, books and documentation. Finally, there are many qualified Java developers.
Java programs, like many other high-level languages, are portable between operating systems and processor architectures ("Write once, run anywhere"). In practice, for example, Android applications work without recompilation on any architecture (Android supports ARM, ARM64, x86, x86-64 and MIPS).
Unlike low-level languages ​​like C and C ++ that use manual memory management, in Java memory is automatically controlled by the runtime environment. The Java program does not even have direct access to memory, which automatically prevents several classes of errors, often leading to falls and vulnerabilities in programs written in low-level languages-no "hanging links" (because of which use-after-free occurs), dereference of a null pointer (if you try to do this, you throw out the nullPointerException), read uninitialized memory, and go beyond the bounds of the array.
Using a full garbage collection (in comparison with automatic reference counting) frees the programmer from all problems and complexities with cyclic references and allows to implement even more advanced (advanced) dependencies between objects.
This makes Android development more enjoyable than development using low-level languages, and applications for Android are much more reliable, including the security point of view.
Running Java is ART
Unlike most other high-level languages, Java programs are not distributed as source code, but are compiled into an intermediate format (bytecode, bytecode), which is an executable binary code for a special processor.
Although attempts are made to create a physical processor that would execute Java bytecode directly, in the overwhelming majority of cases, the Java emulator is used as such a processor - Java virtual machine (JVM). Typically, an implementation from Oracle / OpenJDK called <a rel="nofollow" href=""> HotSpot is used.
Android uses its own implementation called Android Runtime (ART), specially optimized for mobile devices. In older versions of Android (up to 5.0 Lollipop), instead of ART, another implementation called Dalvik was used.
Both Dalvik and ART use their own bytecode format and the native file format in which bytecode is stored - DEX ( Dalvik executable ). Unlike class files in "normal", the entire Java code of the application normally is compiled into one DEX- fileclasses.dex. When building an Android application, Java code is first compiled by the standard Java compiler into class files, and then converted into a DEX file (possibly and reverse conversion ).
And HotSpot, and Dalvik, and ART further optimize the executable code. All three use just-in-time compilation (JIT), that is, compile bytecode at run time pieces of fully native code that runs directly. In addition to the apparent gain in speed, it allows you to optimize the code to run on a particular processor, without abandoning the full portability of bytecode.
In addition, ART can compile bytecode into native code in advance, and not at runtime ( ahead-of-time compilation <tgsrcut >) - and the system automatically schedules this compilation at a time when the device is not used and connected to charging (for example, at night). In this case, ART takes into account the data collected by the profiler during the previous runs of this code (<a rel="nofollow" href=""> profile-guided optimization ). This approach allows you to further optimize the code for the specific operation of a particular application and even for the features of using this application by this user.
As a result of all these optimizations, the performance of Java code on Android is not much inferior to the performance of low-level code (in C / C ++), and in some cases is greater than its .
Java bytecode, unlike conventional executable code, uses the Java object model - that is, in bytecode, explicitly written things like classes, methods and signatures. This makes it possible to compile other languages ​​into Java bytecode, which allows programs written on them to run on the Java virtual machine and to be more or less interoperable with Java.
There are both JVM implementations of independent languages ​​- for example, Jython for Python, JRuby for Ruby, Rhino for JavaScript and the Lisp dialect Clojure - and languages ​​originally designed for compilation into Java bytecode and execution on JVM, the most famous of which are Groovy < tgsrcut>, <a rel="nofollow" href=""> Scala and Kotlin .
The newest one, Kotlin, is specially designed for perfect compatibility with Java and has much more pleasant syntax (similar to Swift), is supported by Google as the official development language for Android on a par with Java.
Despite all the advantages of Java, in some cases it is still desirable to use a low-level language - for example, to implement a performance-critical component, such as a browser engine, or to use an existing native library. Java allows you to call native code via Java Native Interface (JNI), and Android provides special tools for native development - Native Development Kit (NDK), which includes headers, the compiler (Clang), the debugger (LLDB), and the build system.
Although NDK is mostly C / C ++-based, it can be used to write Android and other languages ​​- including Rust < tgsrcut>, <a rel="nofollow" href=""> Swift , Python , JavaScript and even Haskell . Moreover, there is even the ability to port iOS-applications (written on Objective-C or Swift) on Android with virtually no changes.
Classic Unix
The security model in classic Unix is ​​based on the UID / GID system - special numbers that the kernel stores for each process. Processes with the same UID are allowed to access each other, processes with different UIDs are protected from each other. Similarly, access to files is restricted.
In meaning, each UID (user ID) corresponds to its user - at the time of the creation of Unix, there was a normal situation when one computer was simultaneously used by a lot of people. Thus, in Unix processes and files of different people were protected from each other. To allow sharing of some files, users were grouped into groups that corresponded to the GID (group ID).
In this case, all programs that are launched by the user are given full access to everything that this user has access to. Actually, since the user can not communicate with the kernel directly, but interacts with the computer through the shell and other processes - user rights and there are rights of programs running on his behalf.
This model implies that the user fully trusts all the programs that it uses. At that time, it was logical, because the programs most often either were part of the system, or were created (written and compiled) by the user.
In Unix, there is an exception to the access restrictions - UID 0, which is usually called root. He has access to everything in the system, and no restrictions apply to him. This account was used by the system administrator; In addition, under UID 0 many system services are started.
In modern Linux, this model has been significantly expanded and generalized, including the ability to "get some root rights," and implements the mandatory access control (MAC) subsystem SELinux, which allows you to further restrict rights (including rights of root processes).
Everything has changed
For several decades, since the creation of Unix before the creation of Android, the practice of using computers ("calculators") has changed significantly.
Instead of machines designed for parallel use by many users (via terminals - what emulators terminal now emulate), there appeared personal computers designed for use by one person. Computers have ceased to be only a working tool and have become the center of our digital life. With the advent of mobile devices - first PDA, then smartphones, tablets, smart clocks, etc. - this trend has only intensified (because it is relatively inconvenient to deal with working issues on mobile devices).
On such devices are stored gigabytes of personal information, access to which must be protected and limited. At the same time blossomed the market of third-party applications, which the user has no reason to trust.
Thus, in modern conditions, instead of protecting different users from each other, it is necessary to protect other applications, user data and the system from applications. In addition, viruses are widely used, which usually exploit vulnerabilities in the system - to protect against them, it is necessary to additionally protect parts of the system from each other so that the use of one vulnerability does not allow an attacker to access the entire system.
Although some of the Android applications come with the system - for example, standard applications such as the Calculator, Clock and Camera - most users install from third-party sources. The most famous of them is Google Play Store , but there are others, for example, F-Droid , Amazon Appstore , Yandex.Store , Chinese Baidu App Store , Xiaomi App Store , Huawei App Store , etc. In addition, Android allows you to manually install arbitrary applications from APK-files (this is called sideloading).
Like other Unix-like systems, Android uses the existing UID / GID mechanism to restrict access. In this case - unlike traditional usage, when UIDs correspond to users - in Android different UIDs correspond to different applications. Since the processes of different applications run with different UIDs, already at the kernel level, applications are protected and isolated from each other and do not have access to the system and user data. This forms the sandbox (Application Sandbox) and allows the user to install any applications without having to trust them.
To still get access to user data, camera, making calls, etc., the application must get permission from the user. Some of the permissions exist in the form of GIDs into which the application is added, when it receives this permission - for example, obtaining the permission ACCESS_FM_RADIO puts the application in the groupmedia, which allows it to access the file / dev / fm. The rest exist only at a higher level (as records in the packages.xml file) and are checked by other system components when accessing the high-level API via Binder.
A small part of the system services in Android runs under UID 0, that is, root, but most use specially allocated UID numbers, increasing their rights with Linux capabilities if necessary. In addition, Android uses SELinux - the use of SELinux in Android is called SEAndroid - to further limit what actions are allowed to run applications and system services.
Usually, Android does not give the user direct access to the root account, but in some cases, he has the ability to get this access. How it happens, why it is needed, and what dangers it threatens, I'll tell you later.
KlauS 21 october 2017, 10:51
Vote for this post
Bring it to the Main Page


Leave a Reply

Avaible tags
  • <b>...</b>highlighting important text on the page in bold
  • <i>..</i>highlighting important text on the page in italic
  • <u>...</u>allocated with tag <u> text shownas underlined
  • <s>...</s>allocated with tag <s> text shown as strikethrough
  • <sup>...</sup>, <sub>...</sub>text in the tag <sup> appears as a superscript, <sub> - subscript
  • <blockquote>...</blockquote>For  highlight citation, use the tag <blockquote>
  • <code lang="lang">...</code>highlighting the program code (supported by bash, cpp, cs, css, xml, html, java, javascript, lisp, lua, php, perl, python, ruby, sql, scala, text)
  • <a href="http://...">...</a>link, specify the desired Internet address in the href attribute
  • <img src="http://..." alt="text" />specify the full path of image in the src attribute