Android is an open-source project, which has a bunch of cultural and economic consequences. I’m going to ignore those today, and describe how I use the source code to get work done.
Getting the Source · Before you can do anything with the source code, you have to go get it. This could be a daunting task if you’re not familiar with what a “case-sensitive filesystem” is, or how to use the git distributed version control system.
Fortunately, over at source.android.com there are step-by-step instructions that’ll let you get by even if you’re only lightly acquainted with all that stuff.
I’d advise you, if you’re doing anything substantial with Android, to go grab that source code. It takes less than 10G of disk space, and if you haven’t done this kind of thing before, you might find that it feels empowering.
Learning By Example · The Android SDK documentation is, by the standards of commercial APIs, pretty good. By the standards of open-source projects, it’s excellent. But it’s not complete. It’s very hard to combine reference rigor (describing exactly what some method does, with tutorial hand-holding (teaching why and how you’d go about using that method).
In a lot of cases, no documentation can substitute for the experience of looking at how someone who knows what they’re doing has used the piece of the framework that’s puzzling you. That’s what the Android source is good for.
Unfortunately, it can be a little tricky. I just checked a (behind-the-firewall) snapshot of the recent source code, and discovered that it contained 20,989 “.java” files and 6,992 “.xml” files. So, finding what you’re looking for can take some work.
Basic Tools · There are a variety of tools around for doing this, but if you’re working on a Mac or a Linux box, the built-in command-line shell has what you need. Suppose, for example, that I wanted to figure out the use of CallLog.Calls, which exists to help you work with entries from the phone log. I have a little shell script around called “jgrep”, so I hop over to the top of the source-code directory and say:
jgrep CallLog.Calls
Here’s what’s inside that “jgrep” script:
#!/bin/sh
find . -name '*.java' -print | xargs grep "$@"
[Update:] Hah, Jim Ancona in the comments below points out that the Android source tree comes with a pre-built version of “jgrep”, which is called... “jgrep”.
If I run it against my current
source-code snapshot, I get 40 results, of which one is the Compatibility Test
Suite, 5 are in the android.providers
package, and the rest are
in the Google-provided apps.
There’s plenty of information in there to help you to figure out how other people have used this class, if you should, and if so, how.
Getting Fancy · There are lots of times I don’t use “jgrep”, because I want to do something a little fancy, like exclude test-suite files, or look at XML files, or in one step edit all the files that match what I’m looking for.
If you want to do this, you’re just going to have to get comfortable with the programs that are used to build jgrep: find, xargs, and grep. They all have very complete online reference documentation, and the Web is full of examples. I suspect that anyone who’s spent any time as a developer on a Unix-derived platform (Mac, Linux, Solaris, *BSD) will have some feel for find and grep, but quite a few people have never been near xargs. It’s used mostly like this, with find; it takes the potentially thousands of filenames that find prints out and passes them as arguments to whatever other command you’ve provided, in this case grep; it calls grep as many times as necessary and as few as possible to get through all the filenames. The trick is, it knows how many filenames you can pass to a command without anything breaking.
Other Alternatives · I suppose if you loaded the whole Android Open-Source Project into some IDE like Eclipse or NetBeans or IntelliJ, you could get its “Find Usages” tool to find what you want. Has anyone done that?
This find/grep pattern is common enough that there’s a combo tool called “ack” which may already be installed on your computer. You can say:
ack --java CallLog.Calls
My problem with ack is that it’s much slower than just using the built-in tools from the command line. Like 1:40 as opposed to 0:38 for that command on my Mac.
find on my Mac is supposed to have an
-exec grep pattern {} +
argument which has the same effect, but it seems to
Just Not Work for some reason I can’t figure out.
The Next Step · After you’ve found Android code that does something along the lines that you were planning to do, the next step is to... steal it! That’s what it’s there for. Well, since the license invoked at the top of every source file makes it clear that you can copy and re-use the source, it isn’t stealing. But it is what makes open source great.
Comment feed for ongoing:
From: MikeHostetler (May 11 2010, at 10:59)
You need to add ACK to your mailbox. Seriously. Get it at :
http://betterthangrep.com
And the domain is no lie. You will gladly throw away your jgrep script.
[link]
From: Scott Plumlee (May 11 2010, at 11:02)
Just had to laugh at this: " It takes less than 10G of disk space..."
My daughter will probably have a terabyte of her own data before she's 3.
[link]
From: Steve (May 11 2010, at 11:08)
find . -name '*.java' -exec grep CallLog.calls {} +
Don't forget the {} and the + - which are certainly, odd things to have to type on the command line, but find has this historically screwy syntax.
[link]
From: katre (May 11 2010, at 11:10)
Looking at the code, it's licensed under the Apache License, version 2.0. I'm not a lawyer, and I've never played one on TV, and my only knowledge of the subject is that it's a huge flaming tarpit that will suck me down to hell.
So, if I do browse the Android source (licensed under the Apache License, Version 2.0), and I grab a nice big chunk of code to iterate over calls, and stick that in my application, what are the licensing implications for my app? Does it change if my app is also Apache License? What if it's GPL? What if I'm keeping it closed-source and selling the app on the Market?
I'm sure there are good answers to these questions out there, but I'm not sure where to look, just that I'm very worried that looking at the Android code like this may cause future legal problems for my app.
[link]
From: Romain Guy (May 11 2010, at 11:16)
Tim, many Android engineers work with either Eclipse or IntelliJ and tools like "Find Usages" are extremely valuable. I would actually dare say that using an IDE is the best way to find your way around the Android code base at first. The ability to easily navigate to declarations, definitions, find subclasses, methods, usages, etc. is very helpful. And even after a few years spent on the project I still use these tools daily to find what I'm looking for :)
[link]
From: Jim Ancona (May 11 2010, at 11:33)
There is a jgrep (along with several other *grep variants) included in the AOSP source. From the root of the source distro (e.g. ~/mydroid) do:
source build/envsetup.sh
Then type 'help' for a list of commands.
I particularly like sgrep, which searches just about every type of source or build file in the tree.
[link]
From: Tkil (May 11 2010, at 11:40)
You might also take a look at "git grep".
[There's also an unclosed paren in your first or second paragraph...]
[link]
From: PJ (May 11 2010, at 11:50)
ctags might be of use to you. Or an IDE like Eclipse which does the equivalent thing.
[link]
From: davidm (May 11 2010, at 11:55)
I'm quite comfortable in the shell but can't imagine living without an IDE's ability to browse large codebases easily.
[link]
From: Nathan (May 11 2010, at 12:22)
find/xargs has a pitfall that it will choke on files with spaces in the filename. Using -print0 as an argument to find and -0 as an argument to xargs will circumvent this problem.
Also, you spelled 'hop' as 'hope'. Sorry to play the role of typo police.
[link]
From: KevinGaudin (May 11 2010, at 12:29)
I personnaly use the GitWeb integrated grep search :
https://android.git.kernel.org/?p=platform/frameworks/base.git&a=search&h=HEAD&st=grep&s=CallLog.Calls
[link]
From: Matěj Cepl (May 11 2010, at 12:42)
Of course it should read
find . -name '*.java' -exec grep -H "$@" '{}' \;
Or does this work just with GNU tools and somebody who used to live in the archaeological Solaris cannot use it. Doesn’t Mac OS X use something more recent as well?
[link]
From: Z (May 11 2010, at 12:44)
If you have GNU grep (Mac and Linux do, as well as many other systems) you can use:
grep -r --include '*.java' CallLog.calls .
Still, I agree with other comments, an IDE such as Eclipse is much better to explore the Android source code.
[link]
From: huh (May 11 2010, at 12:56)
GNU grep:
grep -r --include=*.java --include=*.xml CallLog .
Use the --include switch from within emacs:
M-x grep
...to get a buffer containing all matches. Click or press enter on that line to jump to that file.
[link]
From: Paul J. Davis (May 11 2010, at 13:00)
For the find -exec syntax you need a trailing semicolon to get it to run.
[link]
From: Andy Lester (May 11 2010, at 13:37)
I'd love to track down why ack is slower than grep (1:40 vs. 0:38) for you. I hear stories of ack being slower for some people but have never been able to track down why, or even reproduce it. You can email me at andy at petdance.com if you've got any insight or examples you could share.
[link]
From: Chris (May 11 2010, at 14:16)
On my mac, I can run the following find/grep/exec command just fine:
find . -name "*.java" -exec grep function {} /dev/null \;
[I use the 2nd parm for grep so that it will print out the file name; also don't forget the escaped semi-colon at the end]
[link]
From: Beat Bolli (May 11 2010, at 15:33)
Guys,
BSD find(1) supports the form "-exec utility [args...] {} +" with the plus sign, which means to substitute with as many filenames as possible for each invocation. This behavior is similar to that of xargs(1). The advantage is twofold: less forks and no need to escape the plus sign from the shell.
[link]
From: Mike Hayes (May 11 2010, at 15:46)
cscope is worth a look if you want to stick with the shell for browsing projects. http://cscope.sourceforge.net/
[link]
From: John Cowan (May 11 2010, at 16:53)
katre: The answer to all your questions is "No problem". Just don't remove the copy of the Apache license from your source (it has to be physically present, even if it's not the controlling license for your program as a whole), or claim that the Apache Foundation is endorsing or is responsible for your product, and all is good.
You may have a problem with mixing the Apache License with the GPL 2.0, but the GPL 3.0 is definitely safe; anything that says "GPL 2.0 or later" is GPL 3.0 in effect.
I am not a lawyer; this is not legal advice. But it's not the unauthorized practice of law, either.
[link]
From: Carey (May 11 2010, at 17:41)
Although my knee-jerk reaction was that the + should be a \;, that is in fact part of the standard Unix find: http://www.opengroup.org/onlinepubs/9699919799/utilities/find.html
[link]
From: Jacek (May 13 2010, at 07:04)
I just found out on my mac that while in "find -exec {} \;" the {} can be placed anywhere, with the plus it has to be placed at the end, so "find . -exec grep pattern {} /dev/null +" doesn't work, but "find . -exec grep pattern /dev/null {} +" does - I also put the /dev/null there because I can never remember the switch to make grep print the file names.
Btw, Tim, please add the link to add a comment also *after* the comments.
[link]
From: Cesar Rivas (May 22 2010, at 11:11)
tim,
I really enjoyed reading your post. The way I learn to program is to review code from others, who are smart enough to open source it. I'm graceful to them.
I have a question wondering in my mind for a while, even before you joined Google or even Oracle acquired Sun. The Android OS is open source, but the Android applications as the gmail app is not open source. Does google plan to open source all of the android apps? or am I wrong? As a whole, the open source form of android is similar to the open source from apple (i.e. the some parts of the source is open but others, as the apps or the GUI are close source) ? As an example, apple opensourced webkit, but not safari.
I would love to heard your comments.
Cesar
ps. by the way, i liked a lot your 'The Setup' interview!
[link]