Undocumented Territory

What happened was, there was an irritating little bug in my LifeSaver app. Which turned into a real problem, since I was using an undocumented API. The story of the bug’s death might be useful in giving a feeling for the 21st-century open-source world.

Background · LifeSaver reads your phone-call and SMS logs and pushes them into the cloud, where they stick around for a couple of hours; the idea is that when you get a new device, LifeSaver can also pull them back out of the cloud scratch-space and load them into the new phone’s logs .

For the phone log this is easy, because there’s an official ContentProvider you can read and update. The equivalent interface for the SMS history has never been documented. Doesn’t mean you can’t use it, doesn’t mean anyone will get mad at you if you do; but there are two bad things that can happen. First, the Android group can change it in a future release and break any software that relies in it. Second, it might not be in the CDD and CTS, so the hardware builders aren’t required to maintain compatibility and your code might not run on some otherwise-compatible devices.

Not that this stops anyone — including me — from using these APIs, if the result you get is useful enough to convince you to go ahead and take your chances. I wrote about this sort of situation on the official Android-Developers blog in New Public APIs in ICS.

My Bug · It turns out that if you’d migrated your history from one device to another with LifeSaver, when you opened the Messaging app, you’d discover that all the conversations were dated, not with the timestamp of the most recent SMS, but whenever it was that you ran LifeSaver. All the individual messages had the right timestamp, and as soon as you started sending and receiving more, the conversations sorted themselves out, so it wasn’t a totally horrible bug. But it was annoying; a bad user experience.

Also, it didn’t happen on all devices: Remember that compatibility stuff I was talking about?

The Long and Winding Road · First of all, I had to figure out where the conversation timestamp was coming from. Use the Source, Tim! Well, except for a couple of hours of that, looking at the Messages app and the Telephony provider, failed to uncover any updates to the underlying date field.

If the source fails you, fall back on an energized community of developers. In other words, type approximations of your problem into Google; which pretty quickly led me to an answer on stackoverflow.com: ”On many phones this cannot be avoided as the messaging application is displaying the date time from the threads table which date time is build from an insert trigger. Some phones will allow you to update the threads provider but this will not work on all platforms.“

[This is a typical experience. I’m only a part-time app developer, but I have never once failed, when faced with a problem, to find good solid help in some combination of the source, Google Groups, and community sites like Stackoverflow. Not once.]

Once I knew I was looking for a trigger, it didn’t take me that long to turn up the code up. Which in turn showed me the SQL idioms used to update that table. Which in turn suggested a bunch more find ... | xargs grep incantations.

Which revealed that when you delete an SMS from a conversation, it has to go update the timestamp based on the most recent message. And as a side-effect of reading this code, I knew which field was used to sort messages into conversations.

So, release 2.1 of LifeSaver, when it jams a bunch of cloud-sourced messages into the database, keeps track of which conversations they’ve gone into, then goes back and, for each one, inserts a dummy message into it then immediately deletes it. Problem solved. Yes, I suspect this is horribly, egregiously, inefficient. But most people only run LifeSaver once per device they own, and I bet the database inefficiency is imperceptible compared to the delay of fishing all those saved messages out of the App Engine cloud.

Is This Evil? · This kind of undocumented-API fiddling isn’t that common in the Android developer ecosystem, because as the years go by the engineering team has got around to documenting almost all of the pieces of the platform you might want to use. But it’s not unheard of at all.

I’m OK with it in this case. LifeSaver doesn’t track user IDs, but it does track some stats: in the last couple weeks it’s been used to save around 300,000 phone calls and messages, and people seem to find value in that; it’ll probably be in the millions soon. (By the way, this is costing less than $1/day on App Engine, using JRuby code with no real optimizations; seems fair to me.)

On the other hand, it could fail when Android’s J release hits the streets or on a perfectly reasonable phone from a first-rate builder; and there’s not much I could do. In this case, I can live with myself, since the worst-case scenario loses nobody’s data, just removes a convenience function.

And in fact, should J change the APIs, it’s not that hard to patch up your app to use the new ones selectively just on the new release, so nobody is even inconvenienced.

Anyhow, there’s one not-evil thing that has to be done: Post a better answer on stackoverflow.com.

And, as always, be grateful that I’m living on an open-source planet, and part of a community that shares knowledge freely, for free, for its own sake.

---

---

Contributions