Here’s something that I wish more people involved in designing consumer software products realized - multiple choices inside a product prevent the user from using the product and make him unhappy. Let me explain why.
Continued – http://softwaredev.quora.com/Offering-Choices-Considered-Harmful
There is a little something that most people who are in the business of creating software understand, but outsiders seldom know. It is this – in software development you cannot get things right the first time.
Continued - http://softwaredev.quora.com/When-in-Doubt-Iterate-Faster
Why does one write? So that others read. And react to what you have written. Now while I get a fairly steady volume on this site, it is nothing to write home about, which is not surprising given that I rarely post anyway.
Of late, I have been thinking of posting more often – mostly about my learnings in building software products. But you know, if you write you want people to read. Now coincidentally, Quora has just recently launched a blogging platform. I have been in love with Quora since a while. It has a highly knowledgeable audience, which is very engaged. And it is viral – as people comment and upvote your content, more people get to discover it.
And therefore I have decided to switch over my writing to Quora. The tech content would go on http://softwaredev.quora.com/ and the non-tech ones on http://www.quora.com/Vineet-Gupta/Posts. Or at least that’s the plan.
My new year has started on an interesting note – I received an email today morning from Microsoft indicating that our App Hub annual subscription has been cancelled:
This was surprising since I most certainly did not cancel the subscription. Note that the mail provides no context as to how this cancellation took place – no tracking id, order-id, issue-id, etc. It just says that it is confirmed that the subscription is cancelled.
I was worried that this may compromise our ability to publish updates to Talk.To for Windows Phone. In fact, just two weeks back I had renewed subscription to our Windows Developer account:
So I quickly logged into the marketplace to check what’s wrong and saw that there are two subscription options there:
The Windows Store Developer account was valid, but the App Hub Annual Membership was expired. Do I need one or both? If one, which one? Note that there is not much to distinguish between the two:
App Hub Annual Membership:
Be a part of a vibrant community of app and game developers for Windows Phone. App Hub has the tools, the content, and the community you need to create and publish amazing apps and games for Windows Phone 7. With an annual subscription of 4500 INR (no taxes charged), your subscription enables you to publish Windows Phone apps using Silverlight or the XNA Framework to the Windows Phone Marketplace.
Become a Windows Store developer, and put your apps at the fingertips of millions of Windows 8 users all over the world. This account gives you access to all the tools you need to submit and publish your app on the Windows Store, plus detailed reports that you can use to track the performance of each app.
So I contacted support, where I was finally rescued by the helpful Jeremiah:
Key takeaways:
Now apparently Microsoft had announced some changes in Aug 2012 of App Hub transitioning over to Windows Phone Dev Center. I vaguely recall the announcement, but it seemed more like a branding change – I did not realize that it would lead to mails regarding account cancellation and such.
What does not help is:
1) The mail is a confirmation of subscription cancellation with no reference to how the cancellation took place, leave alone mentioning that this is part of an automated process and in what scenarios should a customer take some action.
2) On the Dashboard, there is a message that says that Subscription Status is Approaching renewal
3) If you click on that link, it takes you to Account Summary where you are prompted to Renew your subscription.
4) However, you have just renewed the windows developer account subscription and it shows correctly under https://commerce.microsoft.com/PaymentHub/Subscription?accountid=<your id> as being 11 months away!
5) So you think that perhaps you need both subscriptions since in any case the descriptions are similar – you cannot tell which one you do not need.
Could have been much better handled.
I recently upgraded to Windows 8, installed cygwin, copied my keys and config files, etc. only to find that things stopped working.
Try ssh [email protected] and you get:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0660 for '/home/Vineet/.ssh/id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /home/Vineet/.ssh/id_rsa
Permission denied (publickey).
You check permissions and find out:
-rw-rw---- 1 Vineet None 1704 Dec 23 22:57 id_rsa
So you run:
$ chmod 600 ~/.ssh/id_rsa
You expect the permission on the file to change, but it does not. The reason is that the group in cygwin is set to ‘None’ whereas it should be set to ‘Users’
The quick fix is to change the group on all existing entries:
$ chgrp -R Users ~/.ssh.
Now run:
$ chmod 600 ~/.ssh/id_rsa,
This time the permission does change:
-rw------- 1 Vineet Users 1704 Dec 23 22:57 id_rsa
You can now ssh successfully. However, every time you create a new file, the group is set to None and you keep running into the issue of wrong permissions. Here’s how you can fix the root cause:
1) Check the group Id of the ‘Users’ group in /etc/group – it is the third field on the line starting with ‘Users’:
$ cat /etc/group | egrep '^Users:' | cut -f3 -d':'
In my case the value was 545
2) Check the primary group for your user in /etc/passwd – it is the fourth field on the line starting with <your username>
$ cat /etc/passwd | egrep '^Vineet:' | cut -f4 -d':'.
In my case the value was 513 (which was the value for the ‘None’ group in /etc/group).
3) Change this value to the one found in the previous step for ‘Users’ (545 in this example)
4) Exit cygwin and restart
Create a new file and the group should now be set to ‘Users’ by default. You should no longer get any permission errors.
Figuring out all this and fixing it took me well over 2 hours and all because of a silly bug in cygwin. I hope they fix it soon.
I just came back from IIT Guwahati and discovered that Dennis Ritchie has passed away
Most people would know of Dennis Ritchie as the person who invented C and Unix. What most folks do not realize is how profound C and Unix were when they were created and what amazing impact they have had on the industry.
Consider this – before C, you had to program directly to the instruction set of the machine you were programming on. Today when most hardware happens to be x86 or ARM, this seems simple, but back then, the variety of hardware was far more. And this hardware was different not just in terms of instruction sets, but also in terms of information representation, memory addressing, etc. Sure enough there were languages like Fortran and COBOL which could do math and data, but there was no general purpose language which could produce programs that could compete in terms of performance with custom code written for the hardware. Writing a portable program, much less a portable operating system was un-imaginable.
And yet, Dennis Ritchie imagined just such a world and invented a highly performant, high level programming language and then used that to write an operating system which could be ported to any hardware. This one change meant that programmers did not have to create a new OS for every new piece of hardware that got thrown at them – they could take for granted a set of tools, calls and programs, as being available and could spend time going up the value chain and doing more interesting things than just re-inventing the wheel. The result was a spurt of innovation and growth which allowed the industry to grow exponentially into a myriad different directions leading to the world we are in.
If you are a programmer, you owe a lot to Ritchie, whether you have ever programmed in C or worked on Unix or not.
[Edit: Bjarne Stroustrup expresses the same sentiment way more eloquently: http://herbsutter.com/2011/10/12/dennis-ritchie/
Most of us are happy to settle for incremental change. People like Ritchie don’t, and that’s what sets them apart.]
Steve Jobs passed away today – I woke up and read the news on my iPhone, like much of the world.
I never thought I would be so affected by the passing away of someone I never knew personally. But if you are in the world of tech and developing consumer products, it is not possible to have not thought about life as Steve Jobs – how his brain functions, how he figures out what should be built, how he sets the bar, how he hires and build teams, how he delves upon every aspect of the product, … you wish you had 0.1% of his ability, you wish you could be more like him, since being him means creating profound impact, and that is what you are really here for.
And this I believe is his true legacy. The products he helped create would have a shelf life of maybe a decade, Apple as a company would be perhaps super successful for another 2-3 decades, but what would inspire generations of current and future entrepreneurs, programmers and designers is the way he set about building products:
This approach to building products is his true legacy, and for that, the world would be a better place.
As I have mentioned in earlier posts, we are building a x-platform mobile chat client. One of the platforms is iPhone and that means that the team is having to learn Objective C, and therefore C. Last couple of days, I have been spending time helping my team understand and appreciate C better. Yesterday, we got really deep on figuring out how to read a C declaration and it was a lot of fun taking people thru the rules and examples of how to read some complex C declarations. I am summarizing the key aspects over here.
PS: Could not cross link to references since there is no authoritative version of the C-99 standard in an HTML form – the standard docs are all in PDF.
Understanding a C declaration requires that we understand what elements can a C declaration have. While I will describe the overall syntax over here, I will not go into semantics:
declaration: declaration-specifiers init-declarator-listopt ; declaration-specifiers: storage-class-specifier declaration-specifiersopt type-specifier declaration-specifiersopt type-qualifier declaration-specifiersopt function-specifier declaration-specifiersopt init-declarator-list: init-declarator init-declarator-list , init-declarator init-declarator: declarator declarator = initializer
This is what the above grammar is saying:
As per the grammar defined above, a declaration starts with a declaration specifier. A declaration-specifier can be:
auto, static, extern, register and typedef. Default is auto, so if nothing is specified, auto is assumed.intvoidcharsigned charunsigned charshort, signed short, short int, or signed short intunsigned short or unsigned short intint, signed or signed intunsigned or unsigned intlong, signed long, long int or signed long intunsigned long or unsigned long intlong long, signed long long, long long int, or signed long long intunsigned long long or unsigned long long intfloatdoublelong double_Boolfloat _Complexdouble _Complexlong double _Complexconst, volatile, restrictinlineWhat follows a declaration-specifier is a init-declarator-list – a comma separated sequence of declarators that may optionally be initialized. A declarator [2] is defined as:
declarator: pointeropt direct-declarator direct-declarator: identifier ( declarator ) direct-declarator [ type-qualifier-listopt assignment-expressionopt ] direct-declarator [ static type-qualifier-listopt assignment-expression ] direct-declarator [ type-qualifier-list static assignment-expression ] direct-declarator [ type-qualifier-listopt * ] direct-declarator ( parameter-type-list ) direct-declarator ( identifier-listopt ) pointer: * type-qualifier-listopt * type-qualifier-listopt pointer type-qualifier-list: type-qualifier type-qualifier-list type-qualifier parameter-type-list: parameter-list parameter-list , ... parameter-list: parameter-declaration parameter-list , parameter-declaration parameter-declaration: declaration-specifiers declarator declaration-specifiers abstract-declaratoropt identifier-list: identifier identifier-list , identifier
We can break down the above grammar the same way we did for the grammar of a declaration, but I’m gonna skip that and give some examples of declarators, just to give an idea of what declarators are supposed to be:
i – An identifier (and hence a direct-declarator)i, j – A list of identifiersi = 10 – An Identifer followed by an assignment expression*p – A pointer declarator* const p – A pointer declarator with a type-qualifer (const)a[10] – An array declarator with a constant sizea[*] – An array declarator with a variable sizea[] – An array declarator with an unspecified size – the size will need to be defined somewhere elsef() – A function declaratorf(void) – A function declarator with no parametersf(int i) – A function declarator with a parameterf(int i, int j) – A function declarator with a parameter-listNow that we have developed an idea of declaration-specifiers and declarators, we can see that the overall structure of a declaration is of the following form:
However, not all combinations are valid. The following are not allowed:
f()[] // function can't return an array f()() // function can't return a function a[]() // array can't hold a function
The following, however, are allowed:
(* f())[] // function returning pointer to an array (* f())() // function returning pointer to a function (* a[])() // array holding pointers to functions
If you are flummoxed by the last three examples of combining pointers, functions and arrays, you are not alone. Such declarations can look scary till you develop a technique to start deciphering them. In order to do that however, we need to know the basic order of precedence
Here are some examples of how this order applies:
| Declaration | Same as | Not Same as |
int * f(); | int * (f()); | int (* f()); |
int * a[]; | int * (a[]); | int (* a[]); |
Now the way to use these rules is as follows:
If you did not realize this on reading the above description, what we are really doing in steps 2 and 3 is to convert the C declaration into a postfix expression by taking into account the lower predence of the asterisk. This same approach is used in the cdecl program given in K&R.
This will become clear with a few examples:
int * f();
The postfix expression is:
f () * int // "f is" "a function returning" "pointer to int"
Here are a few more examples:
int * a[10]; // postfix expression: a [10] * int // "a is" "array of 10" "pointers to int"
int (* a)[10] // postfix: a * [10] int // "a is" "pointer to" "array of 10" "int"
int **p; // postfix: p * * int // "p is" "pointer to" "pointer to" int
int **p[10]; // postfix: p [10] * * int // "p is" "array of 10" "pointer to" "pointer to" int
int *f(); // postfix: f () * int // "f is" "function returning" "pointer to" int
int (*f)(); // postfix: f * () int // "f is" "a pointer to" "function returning" int
int (* vtable[])(); // postfix: vtable [] * () int // vtable is an array of pointer to function returning int
int (* a[])(int, int); // postfix: a [] * (int, int) int // a is an array of pointer to function returning int // and taking (int, int) as parameters
The way you apply a type-qualifier (const, volatile) is that:
Examples:
const int * p; // postfix: p * const int // p is a pointer to a const int
int const * p; // postfix: p * const int // p is a pointer to a const int
int * const p; // postfix: p const * int // p is a const pointer to int
int * const * p; // postfix: p * const * int // p is a pointer to a const pointer to int
int * const * (* p)(); // postfix: p * () * const * int // p is a pointer to a function returning a // pointer to const-pointer-to-int
An abstract declarator is a declarator without an identifer. However, lack of an identifier does not mean that we can’t interpret such declarators, since the missing identifier’s position can be determined by the placement of (), [] and * Some examples will help clarify this:
int * // pointer to int
int * [10] // array of 10 pointers to int
int * (*) // function returing a pointer to int and taking no arguments
int (*) [10] // pointer to array of 10 int
int (*) (*) // pointer to a function returning an int and taking no arguments
With this background, lets tackle this gem from Andrew Koenig’s C Traps and Pitfalls (PDF):
(*(void(*)())0)();
Clearly, we are casting 0 to some type here. The type being:
void(*)()
We know this type:
// postifx: * () void // pointer to function returning void
In the next step, this pointer is de-referenced and then called. So what is happening here is that zero is being cast to a pointer to a function returning void, then dereferenced and called!
As I mentioned on my previous post, we are building a x-platform native mobile chat client for Android, iPhone and Blackberry. The earlier post was about the Android platform, this one is on Blackberry.
After having gone thru the Android documentation and videos from Google IO, I found the documentation on Blackberry to be somewhat disorganized – it is not easy to get the big picture and grasp key concepts. You need to go thru a lot of documentation to figure out how things works together as a system. I also tried to access the materials from http://www.blackberrydevcon.com/2010-on-demand but it turned out that it would require a registration for $ 20. That was a bit of a WTF moment … I mean, you want to make money by selling conference materials? Who does that?? Anyway I proceeded to register on https://www.eventreg.com/pls/pg_rim/reg_rim.reg_login?icpid=&ipr_code=ONDNON and after filling two lengthy forms, was unable to make a payment despite getting a registration id. I wrote to [email protected] with my registration id (this was on 3rd Mar), but am yet to hear from them. Rather discourgaing for somene starting off on a new platform.
The Blackberry platform consists of the following key components:
This post is specific to Java development for the Blackberry phones. Moreover, it is specific to Blackberry 5 since 75-85% of app downloads from the App World are from Blackberry 5 devices.
Native apps on the Blackberry phone are Java apps that run on RIM’s JVM implementation. This Java API is based on Java ME with Blackberry adding its own extensions. If you are new to Java ME like I am, here is a quick review:
Java ME (Micro Edition) is Java targeted for small devices. It is a collection of specs and technologies to address the needs of providing a programming interface to small devices. Since there is a large variety in the types of devices and their capabilities, it is quite likely that a device manufacturer would implement some specs and not others, leading to API fragmentation and hence non-portable code. To address this, Java ME defines capabilities for devices which are organized into Configurations, Profiles and Optional APIs:
(Organization of the Java ME technologies – source: http://developers.sun.com/mobility/getstart/articles/survey/)
Based on the configurations, profiles and optional APIs, Java ME is able to address a wide variety of small devices including Handsets, Smart-cards and embedded devices:
(Map of the Java ME Universe – source: http://developers.sun.com/mobility/getstart/articles/survey/)
A device implements a stack consisting of a Configuration, Profile and Optional APIs. For wireless devices like mobile phones, the stack is based on CLDC, MIDP and some optional APIs. An application targeting the MIDP profile is called a MIDlet.
(Stack for wireless devices. Source: http://www.oracle.com/technetwork/java/javame/tech/technology-139316.html)
As one can imagine, despite the CLDC and MIDP being defined, a lot of commonly required functionality and the way it is provided remains un-defined, thus introducing scope for API fragmentation and un-portable code. This was addressed thru JSR 185 – Java Technology for the Wireless Industry – a specification that addresses fragmentation by defining key APIs and code portability issues by clarifying spces and providing a suite of compliance tests. JSR 185 consists of CLDC 1.0 or 1.1, MIDP 2.0 and WMA (Windows Messaging API – JSR 120 and JSR 205), with support for MMAPI (Mobile Media API – JSR 135) being optional:
(JSR 185 – JTWI Stack. Source: http://developers.sun.com/mobility/getstart/)
This was superseded by the MSA (Mobile Services Architecture) specification which defined two stacks: a full stack of 16 JSRs and a subset of 8 JSRs:
(MSA Stacks. Source: http://developers.sun.com/mobility/midp/articles/msaintro/)
Blackberry 5 is based on CLDC 1.1, MIDP 2.0 and is JSR-185 compliant. It also supports the full MSA stack except for JSR 184 (3-D graphics), JSR 180 (SIP) and JSR 229 (Payment). For other JSRs supported by Blackberry 5, see http://docs.blackberry.com/en/developers/deliverables/9095/Support_for_standard_Java_APIs_446981_11.jsp
Blackberry applications are of two types:
Unless your code is targeting mulitple Java ME platforms, it makes sense to build Blackberry Java applications since they tend to be a lot richer than MIDlets. Specifically, if you want the look and feel of applications that RIM ships with a Blackberry (also called “native apps”), then you should build a Blackberry Java application by using net.rim.device.api.ui and not use javax.microedition.lcdui. More differences on the UI capabilities of both frameworks are given on http://www.blackberry.com/developers/docs/5.0.0api/UI-summary.html.
Note that all APIs are always available to all apps. So if your app is using the Blackberry UI, you can always invoke MIDP APIs. If your app is using the MIDP UI, you can always invoke Blackberry APIs, but of course doing so would render the application non-portable. The only exception to this is on mixing UI. The MIDP UI model allows concurrent access while the Blackberry UI model does not and hence mixing UI library calls is unsupported.
In the rest of this post, I would be focussing mostly on the Blackberry Java Application.
A Blackberry application can start in one of the following ways:
This startup is managed by an entity called the Application Manager. An running application can obtain a reference to the Application Manager by calling getApplicationManager(), and then, can use this to:
To start an application, the App Manager creates a new process and a thread within that process and then starts the main() function on that thread with a set of input parameters. The parameters are specified in the application descriptor. It is possible to create multiple descriptors for the same app and associate them with different icons on the screen, This allows the creation of multiple entry points for the same app.
Once started, an application can choose to do a series of operations and then quit, never showing a UI. However, most applications want to stay around and respond to system events, including UI events. For this, one of the threads of the application needs to acquire what is called the Event Lock by calling Application.getEventLock(). Only one thread in the application can acquire this lock – this thread is then called the Event Thread. Typically, the first thread acquires this lock by calling enterEventDispatcher(). Once this method is called, the application starts receiving system events. Note that this does not mean that the application is showing a UI – just that is it receiving all system events, including UI events. So as of now, the application is still running in the background. An application can control whether it wants to continue processing systems events by toggling setAcceptEvents().
All the visible elements in a Blackberry Java Application are of three types:
(Blackberry UI Framework. Source: http://www.blackberry.com/developers/docs/5.0.0api/UI-summary.html)
The way the system works is that the UI Engine maintains a stack of Screen objects. As new screens are created, they are pushed on the display stack. Only the screen at the top receives input events. When a screen is popped from the stack, the underlying screens are drawn as necessary. Note that the Blackberry display stack is not the same as the Android back-stack. The Android back-stack is a system-managed structure, scoped to a task with acitivities from different processes getting interleaved. The Blackberry display stack is a per-process application-managed stack of Screens.
To show a UI, the application extends the net.rim.device.api.ui.UiApplication class (which implements UiEngine and extends the Application class), creates an instance of MainScreen (which is of type FullScreen) and then pushes this instance on the display stack.
class MyApp extends UiApplication {
MyApp() {
MyAppScreen myAppScreen = new MyAppScreen();
pushScreen(myAppScreen);
}
public static void main(String[] args) {
MyApp app = new MyApp();
app.enterEventDispatcher();
}
}
class MyAppScreen extends MainScreen {
MyAppScreen() {
// add fields etc.
}
}
An application terminates when System.exit() is called. This method causes the Blackberry JVM to terminate the process of the caller. For applications that show a UI, the typical way of quitting is that when the close() method is called on a screen, it pops the screen from the display stack, and if the display stack is empty, calls System.exit(). So if the user keeps pressing back and hits the first screen of your application, pressing back once again would by default exit the app. To avoid this, one needs to override the close() method.
As described in the Event-Processing section above, the Blackberry UI model is a single threaded one. That means that you cannot do long-running work on it, and instead need to launch a worker thread. This worker thread can update the UI in two ways:
// on the background thread
synchronized(Application.getEventLock()) {
// update the UI
}// on the background thread:
UiApplication.getUiApplication().invokeLater (new Runnable() {
public void run()
{
// update the UI - runs on the Event thread
}
});There are two ways in which Blackberry applications can communicate with each other:
Blackberry has a complex security model, owing to its roots in enterprise software. When businesses issue their employees Blackberry devices, they also expect to able to exercise control over what the user is able to do on the device. This is made possible thru a policy which is pushed to the device using BES or the Blackberry Desktop Manager. Such policies can limit the resources available to an application, and to handle that gracefully, applications should:
Besides this, RIM has marked a number of APIs as controlled APIs. To access these APIs on the device, you must sign your application using a key or signature obtained from RIM. For details, see http://us.blackberry.com/developers/javaappdev/codekeys.jsp. On the simulators, these APIs are accessible without code-signing.
When you build your app, you create a JAR, alongwith a manifest. Java ME defines certain mandatory and optional attributes that applications should include in the JAR manifest, besides any other app-specific info that maybe present.
However, a Blackberry device cannot directly execute a JAR. It first needs to be converted into a proprietary format file called a .COD file which is essentially RIM’s own version of the JAR. So if you want to run a MIDlet on a Blackberry device, you first need to convert the JAR into a COD file which can be done using the Eclipse plugin or by using the RAPC compiler on the command line. Some more details on using the RAPC compiler are given here.
However, the max size of a COD file is 128 kb with a 64 kb limit on compiled code and 64-kb limit on resource data. In case the compiled code produced is larger than these limits, the compiler breaks the code into mulitple COD files, naming them in increasing numerical order like this:
The compiler then takes the numbered files (called sibling COD files), zips them and names the zip as a COD file (HelloWorld.cod above) – this is called the main COD file. There is a max limit to the main COD file too – it can contain up to 127 sibling COD files, thus imposing a theoretical size limit. For more details, see http://supportforums.blackberry.com/t5/Java-Development/The-maximum-size-of-a-BlackBerry-smartphone-application/ta-p/502534.
Packaging the application for Wireless or Over-the-Air (OTA) distribution requires more work. For this, Java ME defines an additional manifest called a JAD (Java Application Descriptor) that Java ME applications should provide for OTA distribution. The main purpose of a JAD is for locating a JAR and figuring out its size. Besides the JAR URL and JAR size, the JAD must also contain certain other attributes that are also required in a JAR manifest and may contain any attribute that is in a JAR manifest. For details, see here. Blackberry specific attributes for a JAD are given here. Also, for OTA distribution, you need to extract the sibling COD files from the main COD file. For details see here.
Blackberry applications can be distributed in two ways:
One of the products we are building at Directi is a x-platform mobile chat client. Developing it has been arduous – we experimented with Phonegap and then Titanium and found both lacking (see this Quora thread for details). So now we are building this as three native apps, targeting iPhone, Android and Blackberry. In this post I am going to cover the Android platform, and would follow up with iOS and Blackberry.
The following diagram (from http://developer.android.com/guide/basics/what-is-android.html) gives the overall architecture of the Android OS:
The Android OS is basically a Linux 2.6.4 kernel fork. The Linux kernel is used for memory management, process management, security, driver model, shared lib model. Google has added Android specific enhancements: Power management, Kernel debugger, Logger, Low Memory killer, Shared Memory Driver, IPC Driver, etc.
Above the kernel, in the user space, sits the Hardware abstraction layer – this deals with stuff like Radio, Wifi, GPS, Graphics, Audio, Camera, Bluetooth, etc. Basically this defines a set of interfaces that drivers need to implement. These libaries are loaded by the system at runtime on a need basis.
Above the HAL, sit a bunch of libraries – the most important of course is libc – Google has written a port called Bionic libc which is optimized for embedded use – in terms of being small in size (since it is loaded in every process) and fast (since CPU is limited), but also because libc is GPL and Google wanted to protect apps from being GPLed. Besides there is webkit of course and SQLite. Google also provides a Media Framework, OpenGL library, and libraries for UI rendering and audio.
The Android programming environment is based on Java. For this, the OS ships with Dalvik – a custom implementation of the JVM optimized for the embedded environment. Dalivk does not run the Java bytecode, but its own custom bytecode called Dex – Dalvix Executable. It uses runtime memory very efficiently, the bytecode interpreter is highly CPU optimized, and because of this low footprint, the VM can be run in multiple processes – basically each app gets its own Linux process with its own instance of a Dalvik VM. The VM is exposed by a set of core Java APIs which provide the familiar Java library.
All of the above is wrapped in a set of platform services called the Android Application Framework – these services work behind the scenes to provide abstractions like Activities, Packages, Windows, Resources, Content Providers, View System, hardware access, etc.
Like a typical Linux system, in Android, the bootstrapper loads the Linux kernel and starts the init process, which in turn starts various daemons like Android Debug Bridge (adbd), Radio Interface Layer Daemon (rild), etc. After this, init starts what is called the Zygote process. The purpose of this process is to kick off the rest of the Android system and later help in the instantiation of apps. The Zygote process initializes a Dalvik VM instance and loads a bunch of libraries and starts listening on a socket for requests to spawn more processes (with VM instances). As requests come in, it forks to create new processes with VM instances. Copy-on-write is used to maximize reuse and minimize footprint.
(Android Startup Sequence – source: http://sites.google.com/site/io/anatomy–physiology-of-an-android)
Once Zygote is in place, the init process starts what is called the Runtime process. This basically does two things:
At this stage, we have the following processes in place:
After this, the Home Screen or the Idle screen is launched – basically the Activity Manager (which is inside the System Server) sends a message to Zygote to start the “Home” Activity, which causes Zygote to fork into a new process with a Dalvik VM and the Home activity. Now depending on the action carried out by the user, the appropriate app would be launched with the Zygote forlking each time to create a new VM instance inside a new process. So for example, if the user starts the Contacts app, the apt process would be setup:
(Startup processes in Android - source: http://sites.google.com/site/io/anatomy–physiology-of-an-android)
Each new app gets a unique user Id which is opaque to the app, and the system sets permissions for all files in the app so that only the user id assigned to the app can access those files.This makes the system secure since the app only has access to the components that it needs to do its work and nothing else. However, apps may need to communicate with each other and the system services, which are all running in separate processes. This is accomplished thru Inter Process Communication
With its emphasis on isolating apps and services in process boundaries, it is clear that Android requries a lightweight IPC. The IPC mechanism in Android is called Binder and is based on shared memory. Recall that when a process starts it registers itself with the Service Manager. This happens behind the scenes, and on registeration, each process gets what is called a Context object – a reference to the Service manager. Now lets say app A needs to communicate with service B, and the two are running in two separate processes. To do this, app A asks the Context for the Service B by passing the well known name of the service. The Context returns a reference to the service to A, on which A can call a method – say foo. This method call is intercepted by the Binder driver. The driver marshals the object and passes a reference of it to the receiver – B. Note that this is passing by reference, not passing by value, in which the object is serialized. On the side of the service B, the Binder maintains a thread-pool (transparent to the service). One of the threads in this pool receives the incoming call, locates the actual object in the service B and make the call. The return value is then similarly passed back to the caller. The following diagram (from http://sites.google.com/site/io/anatomy–physiology-of-an-android) illustrates this:
The entire operation is synchronous, and the objects are reference counted so that when they are no longer in use, they can be deleted. Since there is no serialization / de-serialization, there is no overhead and therefore this model can also be used to communicate with services that run in your own process without any penalty.
An application in Android is a collection of components. There are four types of components:
As mentioned earlier, each application runs in its own process. By default, all components used by that app also run in the same process, and on the main thread. However, it is possible to make a component of your app run in a separate process thru the manifest file. Thus, an application in Android may span multiple processes.
Android follows a fairly unique model in that there is no single entry point for an application – there is no main() function. Instead, a component in one application can start another application’s component, thus bringing the application to life. This communication across apps happens thru the IPC mechanism descibed above. So while an Activity is owned by an application, it is possible for another application to start it (if the owning application allows it). An example is clicking on a hyperlink in an app opening up the browser. This applies for not just Activities, but other types of components also. In order for this to happen, there are two steps required:
Note that in case the application is already running, the new component would be by default instantiated in the same process.
As mentioned above, IPC happens thru a Context object. So when a component A inside one app needs to activate another component B in a different app, or give it something new to do, it basically uses the Context object to send a message to the other component. In the case of an Activity, Service or a BroadcastReceiver, this takes the form of a what is called an Intent – a passive data structure that defines the operation to be performed for an Activity and a Service, and for a Broadcast Receiver, a defintion of the announcement being broadcast.
Content Providers however are not activated thru Intents. Instead, activation happens on a request from a ContentResolver, which acts as a mediator between the requesting component and the Content Provider.
Consider the following scenario:
Here is what happens in the background:
(The Activity Back Stack – source: http://developer.android.com/guide/topics/fundamentals/tasks-and-back-stack.html)
This approach allows Android to seamlessly transition from one app to the other in a consistent way.
In the scenario described above, assume that when the user was in Compose mail (Activity 3), he decided to call someone and pressed the Home key. This would not unwind the back-stack, but start a new stack. In order to do this, the collection of Activities in the first stack needs to go into the background. This is achieved thru the notion of a Task – a cohesive unit of Acitivities. When a task moves to the background, all activities in the Task are stopped but the back-stack for the task remains intact, so that when the user returns back to the Task, she can resume where she left off. However, to save memory, the back-stack for a background Task is not retained for a very long period and if the user does not go back to the task, the back-stack is cleared except for the root activity.
(Background and Foreground Tasks – source: http://developer.android.com/guide/topics/fundamentals/tasks-and-back-stack.html)
The lifecycle of an activity is affected by its association with other activities, its task and its back-stack. There are three states in which an activity can exist:
(Activity Lifecycle – source: http://developer.android.com/guide/topics/fundamentals/activities.html#Lifecycle)
Note that it is entirely possible that the system, in order to reclaim memory, may destroy an Activity, or even the process in which the Activity was running. However, when the user comes back to the activity (thru the back-stack), you still want to resume at the point the user left. To do this, an activity must save its state. This happens thru the Activity.onSaveInstanceState() method.
(Saving and restoring Activity State – source: http://developer.android.com/guide/topics/fundamentals/activities.html#SavingActivityState)
The Android system may need to kill a process in order to reclaim memory. To ensure that this creates minimal impact on the user expereince, Android ranks processes in a priority order:
Of course it can so happen that a higher priority process is dependent on a lower priority process. In such a case the ranking of the serving process is increased to the same level as that of the dependent process.
A situation where this ranking has a direct impact is this:
When an application is launched and a new process created to host it, the application gets a single thread called the main thread or the UI thread since this thread is responsible for servicing the UI. The way this works is mostly the same for almost any UI implementation – be it desktop operating systems or mobile OS. Basically, the UI thread runs an infinite loop which checks a queue to see if there are any pending UI events. In the case of Android, this concept is formalized in the form of a Looper. The looper loops over a MessageQueue which contains the messages to be dispatched. The actual task of managing the queue is done by a Handler which is responsible for handling (adding, removing, dispatching) messages in the message queue.
(Android Threading – source: http://sites.google.com/site/io/inside-the-android-application-framework)
For the main thread, a Looper is setup by the system. However, you can also associate a Looper with your own thread. Note that the Looper can be associated only with one thread and that association cannot be changed. The way Android ensures this is by putting the Looper on the thread local storage of the thread and not exposing a constructor for the Looper. You get a Looper by calling the static prepare method which first checks the TLS to see if there is already a Looper, and if not creates one. You then call the loop method on it to start pumping messages. Unlike a Looper, mulitple handlers can be associated with a message queue.
So quite obviously, one should not run a blocking operation on the UI thread. If you do, messages would stop getting processed from the Message Queue and your app would become unresponsive. This is what leads to the famous Application Not Responding (ANR) dialog. One solution is decsribed here: create a handler, spawn a worker thread, post results back from the worker thread to the handler, update the UI. But this is a bit cumbersome to do. A simpler solution is to use a AysncTask which takes care of the underlying plumbing.
As mentioned earlier, a Service is a type of component that is used to carry out a background operation, and does not provide a UI. An app component can start a service, and even if the user switches out of the application, the service would keep running. Typical use cases include playing music and downloading a file. Android ensures that a process running a Service is not killed as far as possible (see Process lifecycle above).
Note that a service by default runs on the main UI thread in the same process. This means that if it carries out a blocking operation, your UI would become unresponsive since the thread would not be available to run the Activity. To circumvent this, you should launch a worker thread inside the service.
Recent Comments