Thursday, 22 August 2013

FOTA in Android (Mechanism, Implementation & Behind the scene)
-Firmware updation in Android over the air


What is FOTA?

Firmware Over-the-Air (FOTA) is a method used for update software on mobile phones and tablet computers etc.

FOTA in Android?

In android, FOTA can update current firmware, kernel, boot/system/recovery images, single/multiple application etc.
Major components involved in Android FOTA mechanism are:
  •        FOTA Server with push notification support
  •        OTA Updater application in Android device
  •        Recovery console in Android device

FOTA procedure in Android?

In Android smartphones/tablets FOTA is performed in following way:

  •         Device register to server for push notification of firmware/software updates.
  •         Server sends the update notification to the device.
  •         Device perform polling requests to server for new update.
  •         User performs manual update check.
  •         If update is available, get the update file URL from server.
  •         Fetch update file and save in cache partition.
  •         Verify the update file signature and checksum.
  •         Reboot in recovery mode with update installation command.
  •         Recovery console verifies the signature of update file.
  •         Recovery console executed the updater script from update file.
  •         Device Reboots after installation.



How to implement FOTA in Android (excluding push notification)


1. Create the Cryptographic signature for the OTA signing (required for releasing, not needed for demo)

There are four keys that can be used during development for authentication purposes:
Ø  testkey — a generic key for packages that do not otherwise specify a key.
Ø  platform — a key for packages that are part of the core platform.
Ø  shared — a  key for things that are shared in the home/contacts process.
Ø  media — a  key for packages that are part of the media/download system.
The default AOSP keys are found in the build/target/product/security directory.
(*.pk8 are the private keys, *.x509.pem are the corresponding certificate/public keys)
We can generate our keys with the shell script located at /development/tools/make_key

Procedure to create Keys is as follows:
  • First remove the existing AOSP keys :
  • Create New keys :
  • Verify the created keys :
  • Re-sign the OTA file content after creation :
  • Re-sign the OTA file manually with SignApk.jar :

      2.  If the new update package can have older “build date time” than only we should comment the following line from following file :
(otherwise skip this step):
/build/tools/releasetools/ota_from_target_files
      "script.AssertOlderBuild(ts);"
  • As the mentioned line creates the assert in the updater script for older build and aborts the installation if the update file is having older “build date time”.

     3. Create an FOTA update file with the command
          It will generate the full Android OS OTA update file in the out/target/product/product_name/someotafile.zip

     4. If we need to create any partial/incremental update then we can do with following command:
  •   It will only include the specified packages in OTA update file, its build logic is in build/core/Makefile.

     5. Now upload the zip on OTA server and whenever the update check for your device comes at server
    After verifying device, rom values from the request provide the Rom info in the form of Jason String.

     6. Create an Android application with following permissions:
  • So that application can do following tasks:
    • Perform polling request to server using internet on each time device boots/ manual update check by user.
    • Update can be downloaded to /cache partition.
    • For installation in recovery mode device can rebooted by the application.

      7. Create a Activity with intent filter "android.settings.SYSTEM_UPDATE_SETTINGS"
       This intent is fired from /packages/apps/Settings/res/xml/device_info_settings.xml
  • So that when user navigates to System settings -> About device-> System Updates, our activity can be launched.

             8. Create an AsyncTask, which can be called whenever polling is required to be performed to check the update on server, the doInBackground() method will have pseudo code as follows: 
  •         9. Create a broadcast receiver with intent filter "android.intent.action.BOOT_COMPLETED"
  •         So that when device boots up we can start our UpdateCheckReceiver class, which can Schedule a repeating alarm to send the pending intent to this same broadcastReceiver class to instantiate a new AsyncTask, to perform polling on server, to check the update.

      10. On a valid RomInfo Object received, Start the Downloader AsyncTask to download the update zip file from server, the doInBackground() method will have pseudo code as follows:

  •   11. On successful download of the update zip file from server we must verify the cryptographic signature of the downloaded update file, and then we can install it as follows:
  • It will reboot the device in recovery mode, where recovery console will first verify the signature of the file then execute the updater script of that update zip file, which will install the updates i.e. (FOTA).
    If any problem occurs it will show an android error symbol and wait for user reboot, otherwise It will reboot in to updated android after successful installation.


       Behind the scene


    ·        Releasetools (which creates OTA update file)
    – It Create digitally signed software updates from a target-files-package (TFP)
    -  TFP generated by Android* build system
       We need to substitute testing for production keys inside a TFP

    ·        SW Update UI intent from AOSP Settings application
    -  android.settings.SYSTEM_UPDATE_SETTINGS
    -   Intent launched when user navigates to Settings->About device->System updates

    ·        android.os.RecoverySystem APIs
    – Framework APIs to verify & install SW updates
       Handles verification of OTA update digital signature
    -   Also used to engage Factory Data Reset
    -   RecoverySystem doesn’t write to BCB due to permissions on doing raw block device I/O
    – Writes RC command files into /cache/recovery/command, and reboots into Recovery Console
    – Typical RC commands which may be supplied in the /cache/recovery/command file:
           --send_intent=anystring - write the text out to recovery.intent
        --update_package=path - verify install an OTA package file
        --wipe_data - erase user data (and cache), then reboot
        --wipe_cache - wipe cache (but not user data), then reboot
        --set_encrypted_filesystem=on|off - enables / diasables encrypted fs
        --just_exit - do nothing; exit

    ·        /misc partition
       Very small size, does not contain a filesystem
       Tiny partition used for communication between Recovery Console (RC) and bootloader, and for RC to save state information
       Contains Bootloader Control Block (BCB)
    Ø  command[32]: Commands for the bootloader
    • “boot-recovery” boot into RC instead of Android
    • Other platform-specific commands may be implemented for update tasks that must be done by the bootloader
    • If empty, garbage, or no known commands matched, normal Android boot

    Ø  status[32]: Return status field written by bootloader after performing platform-specific commands
    • No specification, platform-dependent

    Ø  recovery[1024]: Command line for Recovery Console
    • Arguments tokenized by ‘\n’
    • Invalid if first argument not ‘recovery’

    ·        Recovery Console (RC) (Alternate boot environment)
    – Comments at the beginning of bootable/recovery/recovery.c are out of date
    -   A few ways to enter –
    Ø  reboot recovery from a shell
    Ø  RecoverySystem APIs in Android Framework
    Ø  OEMs often implement a bootloader ‘magic key’
    – Verify & Apply SW updates
    – Perform Factory Data Reset
    – Typically controlled by “/cache/recovery/command” file left by Android RecoverySystem APIs
    -   Pictorial interface, no localization
    -   Hidden non-localized menu for manual tasks
    -   Other platform-specific tasks as implemented in Recovery Console UI plug-in
    -   Log files saved in /cache/recovery
    Ø  All stdout/stderr from RC and Updater
    Ø  Edify ui_print() commands
    -   Upon startup, looks for command line arguments in decreasing precedence:
    Ø  Actual command line to ‘recovery’, debug-only scenario
    Ø  BCB.recovery
    -   Android framework RecoverySystem doesn’t write to BCB due to permissions on doing raw block device I/O, So Recovery console (RC) Always copies arguments into BCB.recovery and sets BCB.command to “boot-recovery”
    Ø  Makes sure we keep booting into RC with the same arguments in event of unexpected power loss
    Ø  Don’t rely solely on /cache/recovery/command for this
    -   finish_recovery()
    Ø  Called when requested operations (SW update, factory data reset, etc) are complete, whether successful or failed
    Ø  BCB is cleared so that subsequent reboot goes back into Android
    Ø  Copies all logs to /cache/recovery/
       If no arguments were given to RC, displays error image and waits for menu input

    ·        Bootloader Integration
    -   Linux kernel should write “boot-recovery” into BCB.command and zero out BCB.recovery if “recovery” is supplied as a reboot() argument from android framework
       Implement in a driver via register_reboot_notifier()
    -   Bootloader selects boot image (or other task) based on BCB.command
       BCB.command is persistent; keep booting into RC until RC clears it
       Garbage or zeroed out contents should simply boot into Android

    ·        Updater
       SW update logic, binary inside SW update package
       AOSP implementation runs script in Edify language
       Platform-specific tasks implemented in plug-ins

    ·        updater script
    -   Updater script written in Edify language
    -   Created by python script in AOSP /build/tools/releasetools/edify_generator.py & /build/tools/releasetools/ota_from_target_files
    -   Placed inside update zip file in /META-INF/…/updater-script





    Wednesday, 24 July 2013

    Drive Trace Android Application


    http://ford.challengepost.com/submissions/16641-drive-trace-application?utm_campaign=ford-personalized-fuel-efficiency-app-challenge_20130325&utm_content=submission_visible_in_gallery&utm_medium=email&utm_source=transactional




    Drive Trace application helps driver learn, compare and share information about his driving style and thereby help him to optimize the fuel usage. 

    In on-drive mode, the application interacts with driver through Sync AppLink for trip creation or continuation and records per trip start and end location, fuel consumption, mileage and other details along with real-time vehicle data through OpenXC. It analyzes driving style through concrete algorithms and raises alerts on the go. Alerts can be enabled on configurable frequency to help driver unlearn the driving habits that lead to aggressive driving or over-idling. 

    In off-drive mode, driver can view per trip details and effortlessly compare and analyze mileage, aggression & idling trends for selected trips. The application also supports managing trips and sharing of analysis graphs on e-mail, Bluetooth or on any social networking sites.

    Tuesday, 16 July 2013

    ADB Over WiFi.

    ADB Over WiFi

    Are you an Android developer ? If yes, you certainly use ADB (Android Debug Bridge) to do various things like:

    • Start a shell on the target device.
    • Install/Uninstall some packages/ apps.
    • Monitor the state of the device (memory used, cpu used etc.)
    • Push/Pull some files to/from the target device.
    • Debug Android applications.
    Most developers connect their device to their computer using a USB cable and that sucks because:
    • It is not ergonomic.
    • It is ugly.
    • You probably need that cable to plug your device to an electric outlet while you are at work.
    • You may not have a USB cable at all!
    Anyway, for those of you who want to connect to your device wirelessly using ADB, I have some good news for you: it is possible and it is easy.
    First, let’s talk about how ADB works and then let’s dig into that very straightforward solution (or if you are busy just jump to the second paragraph).
    How does ADB work?
    ADB is a tool that includes three components:
    • A daemon: It is a background process hosted by the target device.
    • A client: It runs on your development machine. This is where you will be issuing your commands.
    • A server: It acts as a Proxy between the client and the daemon. It runs as a background process on machine.
    As you can guess, there is a two way communication going on between the server and the daemon. That communication happens over a medium which can be of two sorts: USB or TCP. Aha, you get it know? If we want to connect to our device over wifi then we just have to switch the transport mode from USB to TCP. Let’s do that!
    The solution: Switching ADB transport mode from USB to TCPIP
    1. Before switching to TCP, we’ll need to grab the IP address of the device. To do that, we can either go to the phone settings or use the command line tool netcfg
      view sourceprint?
      adb shell netcfg
    2. Once you are in possession of the device’s IP address (let’s say it’s: 192.168.65.150), you can switch to TCP mode by doing: 
      view sourceprint?
      adb tcpip
       One thing to remember is that you can only choose a port number within the range of [5555...5585] (the reason being that ADB Server only works with these ports). So for instance you could do: 
      view source
      print?
      adb tcpip 5555
    3. Stop ADB Server (we’ll restart it later by issuing any ADB command): 
      view sourceprint?
      adb kill-server
    4. Set ADBHOST to your IP address: 
      view sourceprint?
      ADBHOST=192.168.65.150 adb devices
    5. Make sure your device is connected
    the last part of the last command should print the device that you are trying to connect to.
    What to do if you don’t have a USB cable from the beginning? (rooting required)
    If you don’t have a USB cable at first, then you need to root your device. (can void warranty)
    Than run the following command on your device: 
          setprop service.adb.tcp.port 5555
    Now you need to stop/restart the adb daemon: 
          stop adbd
          start adbd
    How to switch back to USB transport mode? 
          adb usb
    But there is one caveat
    This only works since Android 4.0 (ICS). If you have to develop/hack on other Android versions then you need to root that device (in order to use ADB in unsecure mode).
    That’s all folks, I hope that you are now using debug bridge over the air :)

    Thursday, 20 June 2013

    [Solved] - Nexus 7 Doesn't Show Up As Media Device (MTP)

    Solution To Nexus 7 Doesn't Show Up As Media Device (MTP)

    1.       Plug the Nexus 7 into your computer with the USB cable.
    2.       Open Device Manager.
    3.       Find "Android Device" near the top and expand the node. Double click on "Android Composite ADB Interface".
    4.       In the properties window that pops up, uninstall the driver (chose to delete the current driver as well).

    5.       Unplug the tablet and plug it back in.

    Friday, 24 May 2013

    Adding New System Service in Android


    To create a custom system service in android following steps are required

    1. Create service in framework
    2. Register service in SystemServer
    3. Create a .aidl interface of the service in framework
    4. Register .aidl interface in build system
    5. Create a manager for the service in framework
    6. Leverage the features of the service from service manager 
    7. Register the service manager in ContextImpl
    8. Register name of service manager in Context (abstract class)
    After performing the above steps we need to build the new SDK for using the features of service through service manager.


    1. Create service in framework

    We need to create a service (Not an android service) in directory /frameworks/base/services/java/com/android/server/ where other system service are placed.

    example code:

    /*DemoService.java */
    package com.android.server;
    import android.content.Context;
    import android.os.Handler;
    import android.os.IDemoService;
    import android.os.Looper;
    import android.os.Message;
    import android.os.Process;
    import android.util.Log;

    public class DemoService extends IDemoService.Stub {
        private static final String TAG = "DemoService";
        private DemoWorkerThread mWorker;
        private DemoWorkerHandler mHandler;
        private Context mContext;
        public DemoService(Context context) {
            super();
            mContext = context;
       mWorker = new DemoWorkerThread("DemoServiceWorker");
            mWorker.start();
            Log.i(TAG, "Spawned worker thread");
        }

        public void setData(int val) {
            Log.i(TAG, "setData " + val);
            Message msg = Message.obtain();
            msg.what = DemoWorkerHandler.MESSAGE_SET;
            msg.arg1 = val;
            mHandler.sendMessage(msg);
        }

        private class DemoWorkerThread extends Thread {
            public DemoWorkerThread(String name) {
                super(name);
            }
            public void run() {
                Looper.prepare();
                mHandler = new DemoWorkerHandler();
                Looper.loop();
            }
        }

        private class DemoWorkerHandler extends Handler {
            private static final int MESSAGE_SET = 0;
            @Override
            public void handleMessage(Message msg) {
                try {
                    if (msg.what == MESSAGE_SET) {
              Log.i(TAG,"set message received:"+msg.arg1);
                    }
                } catch (Exception e) {
                   // Log, don't crash!
              Log.e(TAG, "Exception in handleMessage");
                }
            }
        }
    }


    2. Register service in SystemServer
    We need to register our service in /frameworks/base/services/java/com/android/server/SystemServer.javawhere other system service are registered.
    /*SystemService.java*/
    /*
     * go to function "@Override public void run()"
     * ........
     * Add following block after line "if (factoryTest !=
     * SystemServer.FACTORY_TEST_LOW_LEVEL) {" 
     */

    try {
        Slog.i(TAG, "Starting Demo Service");
        ServiceManager.addService("demo", new DemoService(context));
        Slog.i(TAG, "Demo Service Started");
    } catch (Throwable e) {
        Slog.e(TAG, "Failure starting DemoService Service", e);
    }




    3. Create a .aidl interface of the service in framework
    A service can expose set of functions that can be access by other process/application, by esposing them in .aidl file.
    To expose the functions of our DemoService we need to declare them in .aidl file in any existing directory or any new directory in framework 
    Note: The directory represent the package structure so if we want to add our service manager in any existing package we have to put our .aidl in that directory other wise our new directory.
    (here we will be creating our .aidl in framework's existing directory for simplicity, as creating new directory in framework is little complex which we will discuss later)
    So we need to create IDemoService.aidl in direcotry 

    /*
     * aidl file :
     * frameworks/base/core/java/android/os/IDemoService.aidl
     * This file contains definitions of functions which are
     * exposed by service.
     */
    package android.os;
    /**{@hide}*/
    interface IDemoService {
            void setData(int val);
    }


    4. Register .aidl interface in build system

    We need to register our service's aidl interface in frameworks/base/Android.mk
    Note: here we don't have to add entry of any aidl files for parcelables if we use any in our IDemoService.aidl.

    /*
     * open frameworks/base/Android.mk and add line in red 
     * color mentioned below which contains location of
     * our .aidl file.
     */
    ...
    core/java/android/os/IPowerManager.aidl \
    core/java/android/os/IDemoService.aidl \
    core/java/android/os/IRemoteCallback.aidl \
    ...


    5. Create a manager for the service in framework
    There are two approaches to create the  our service manager either singleton or a normal class.
    Note: If we create a normal class if we use that class from application we will end up with multiple instances of our service manager (still those different instances will contain same static instance of our system service), And than to provide single instance of our service manager we need to register our service manager in /frameworks/base/core/java/android/app/ContextImpl.java and 
    so to optimized and simple we will see with singleton

    Below is the sample code for a Singleton manager class:  

    /**
    * /framework/base/core/java/android/os/DemoProxy.java
    * It will be available in framework through import android.os.DemoProxy;
    */
    import android.os.IBinder;
    import android.os.IDemoService;
    import android.os.RemoteException;
    import android.util.Log;
    /**
     * Use this Singleton class to call the functionality of DemoService
     * @author shridutt.kothari
     * It is a like a ServiceFetcher for the ContextImpl.
     */
    public class DemoProxy {
         private static final String TAG = "DemoProxy";
         private final IDemoService mDemoService;
         private static DemoProxy demoProxy;
     
         /** Get a handle to the DemoService.
          * @return the DemoService, or null.
          */
         public static synchronized DemoProxy getDemoProxy() {
                if(DemoProxy == null) {
                IBinder binder = android.os.ServiceManager.getService("demo");
                        if(binder != null) {
         IDemoService managerService = IDemoService.Stub.asInterface(binder);
                                demoProxy = new DemoProxy(managerService);
                        } else {
                               Log.e(TAG, "DemoService binder is null");
                        }
                }
                return demoProxy;
         }
     
         /**
          * Use {@link #getDemoProxy} to get the demoProxy instance.
          */
         DemoProxy(IDemoService demoService) {
              if(demoService == null){
                throw new IllegalArgumentException("demo service is null");
              }
              mDemoService = demoService;
         }
         /**
          * Sets the value in demoService 
          * @param arg
          */
         public void setData(int arg){
             try{
                Log.d(TAG, "Going to call service from framework proxy");
                mDemoService.setData(arg);
                Log.d(TAG, "Service called successfully from framework proxy");
             } catch (Exception e) {
                Log.d(TAG, "FAILED to call service from framework proxy");
                e.printStackTrace();
             }
         }
         /**
         * Get the binder of IDemoService.
         */
         public IDemoService getDemoService(){
                mDemoService;
         }
         
    }
    Now 3rd party apps can call getDemoService().setData(); method.

    6. Leverage the features of the service from service manager 
    We see in above code snippet setData (int arg) method is to leverage the feature of our custom service, As we have called mDemoService.setData(arg); inside setData method of our service manager (i.e. DemoProxy)


    7. Register the service manager in ContextImpl


    Now applications will be able to get the instance of our service manager with the DemoProxy.getDemoProxy(); But if our service manager is not a singleton class we need to register it in ContextImpl so that applications can get instance of service manger like Context.getSystemService("service_name");

    Now our service, aidl of service, service manager all things are created, so we need to write following code in /frameworks/base/core/java/android/app/ContextImpl.java file in the static block: 

    /**
     * In /frameworks/base/core/java/android/app/ContextImpl.java
     * inside the static block we need to register our ServiceFetcher like this:
     */
    registerService(DEMO_SERVICE , new ServiceFetcher() {
            public Object createService(ContextImpl ctx) {
            return DemoProxy.getDemoProxy(); 
    /**
     * Or we can create new object of it, if DemoProxy is not a singleton class.
     * see other registered Services for example.
     */
    }});


    8. Register name of the our service manager in Context (abstract class)

    Now we need to add entry of DEMO_SERVICE as a constant in file


    /** inside /frameworks/base/core/java/android/content/Context.java. */
    /**
     * Use with {@link #getSystemService} to retrieve a
     * {@link android.os.DemoProxy} for using DemoService
     *
     * @see #getSystemService
     * @hide
     */
     public static final String DEMO_SERVICE = "demo";

    After all the above steps we just need to build our custom 
    sdk and than, we can use the service manager in applications like:


    DemoProxy dm= (DemoProxy) getApplicatioContext().getSystemService(Context.DEMO_SERVICE);