OS Upgrades and Security Patches in the enterprise hardly ever consist of just a simple step. Apps need to be targeted for a new Android SDK, newly introduced settings must be decided on, and mobility extensions may need to be updated. On top of that, there are considerations for the varying environments and device states that must be accounted for. This post will highlight some of the challenges BlueFletch has encountered managing OS Upgrades in hopes to get your team thinking about hurdles that you may experience along the way. Learn how to upgrade your Zebra Symbol Android Device here!
BlueFletch has been tasked with developing a process and deploying OS Upgrades for many large fleets of enterprise Android devices numerous times over the years. Some of the problems are common across the board, and some are unique. In any case, we can group these challenges into three core areas: Network, Device Behavior, and Store Operations.
Networks
- Poor networks: It’s 2019 and we still see sites with slow network connections resulting in unreliable downloads from limited bandwidth. This is especially true and problematic in rural areas, as we experienced when upgrading devices from inland Australia.
- Unreliable downloads: Downloading large files from a remote server occasionally leads to partial or corrupted downloads. We have seen multiple retries needed to successfully and completely download the update package. Ensuring that remote servers are scaled properly or using local servers instead reduces the frequency.
- Network over-utilization: After business hours, systems like POS’s run backup that impact bandwidth during OS Upgrade service windows. It is important to find these windows of limited network utilization and deploy during them to ensure you have reliable download performance. OS Patches can easily exceed 100Mb and when 40 devices concurrently attempt to download, networks quickly go pear-shaped.
Device Behavior
- Persistence storage: An OS Upgrade will wipe everything from the device unless it is designated as persistent and stored in a secure folder. Unfortunately, some clients overuse this folder and the resulting lack of memory will cause nothing to persist. Ensuring only critical applications and settings are included in Persistence will provide a higher likelihood of success. At a minimum, the WiFi settings and MDM Agent must be persisted to apply to all other desired images.
- Loss of proxy: When migrating devices from Jellybean (4.1) to KitKat (4.4), we found that the WiFi profile would remain on the device but wouldn’t carry over the proxy settings. The client required proxied traffic to reach the cloud-based MDM, thus all control over the device was lost.
- AirWatch unenrollment after Upgrade: After an Upgrade, the AirWatch Agent reinstalls and validates enrollment. During a large deployment, we found a bug that affects about 5% of devices. Instead of re-enrolling, the Agent unenrolls the device and the link to the MDM is lost. The device becomes nearly a stock Android device that must be reconnected and enrolled.
- Battery levels: Android requires at least 30% battery level to apply any patches or OS Upgrades. Ensuring the device is in a cradle increases upgrade success rates, but we have seen clients with not enough cradles to house all of their devices. One client didn’t even have cradles at all!
- Non-compliant devices: Devices that are not fully compliant will always be an issue with a large enterprise fleet. How will these handle upgrade flow if they are missing a critical component like Zebra Power Manager? We separate these devices off into a different remediation path and clean them up periodically.
Operational
- Non-business hours: Deploying during non-operating hours becomes tricky when dealing with stores across multiple time zones. Having the ability to phase deployments across local times will maximize service windows and give the most runway for Upgrades. Ideally, sites and devices are already grouped for the waves.
- User interaction (barcode scan): We’ve experienced some scenarios where a process cannot be completed entirely over-the-air. For example, a client needed to connect to a proxied network to receive download and install commands so a StageNow barcode scan was required to add to the network. Barcodes should also be on-hand for remediation activities when a device does not re-enroll correctly or WiFi does not connect.
- End of shift process: Stores should already be cradling their devices on close of business, but if they are not a huge opportunity to service devices in an ideal state is being missed. Enforcing proper device check-in ensures they are charging, close to WiFi, and most importantly – accounted for. Extra communications should be broadcast to cradle during the OS upgrade service window.
- Automation in MDM: An OS Upgrade is rarely composed of just one .zip file to update the operating system. Many times there are driver updates, application updates required, and new settings that must be applied. Being able to automate these through triggers and inclusion rules will reduce breakpoints and minimize service time. A device should be able to have a “one-click” process that is thoroughly tested and escape proof.
- Cleanup of old files: Devices may have gone through upgrades in the past already. Legacy files from prior OS upgrade activities sometimes are not cleared off and space needs to be made for the download of new packages. Surveying device storage ahead of time indicates if files need to be deleted early in the process.
Many of these challenges we’ve experienced can be overcome in a variety of creative ways. However, some required us to write code to handle them programmatically. We decided to combine all the solutions into a single OS Update Tool for Zebra. In my next blog post, I will address how our tooling solves these problems in detail. If you are interested in learning more, feel free to visit our microsite for our OS Update Tool or contact us at info@bluefletch.com
