bunnyhero dev

While playing with the new AVFoundation APIs, it occurred to me that in iOS 4, apps can now easily access the camera with no feedback to the user. Before, apps had to use UIImagePickerController, which shows the iris-opening animation before recording starts, even if you hide the preview image using cameraViewTransform. With AVFoundation’s AVCaptureSession, there is no indication to the user at all that the camera is in use unless the app provides its own. There is no permission alert, nor any LED indicator like a webcam. An app could secretly be recording your face with the iPhone 4’s front-facing camera and sending it to who knows where. I wonder if Apple’s app review team checks for this in some way?

On the other hand, the new APIs make it much easier to integrate non-photo-taking uses of the camera into an app. I could imagine using the iPhone 4’s front camera for non-touch gesture controls or facial expression recognition. Makes me wish I knew something about real time image processing!

iOS 4 added a lot to AVFoundation, including classes and APIs that give you much more control over the iPhone camera. One of the things you can now do with the camera is read the video frame data in real time.

In this post, I’ve created a simple demo that simulates a Twilight-style vampire. In the Twilight series, vampires aren’t hurt by daylight; instead, they sparkle. Yes, sparkle.

Here are a couple of screenshots from the app:

And here’s a low-quality video of the vampire simulator in action.

The app detects the amount of light shining on the phone by doing very simple image analysis of the incoming video frames from the camera. The brighter the image seen by the camera, the more sparkles it draws on the vampire.

So how does this all work?
» Read the rest of this entry «

Like many people with barely-updated blogs, I want to blog more often. I’m not much of a writer but that can only change with practice, right? Plus, I’m a big believer in community and the sharing of knowledge, and I wanted to contribute to that more. But where would I find the writing discipline?

I’ve also been a fan of personal writing challenges for some time. For example, I’ve participated in National Novel Writing Month (NaNoWriMo) for many years, where the goal is to write a 50,000-word novel in 30 days (in November). The concrete and public goal, combined with the camaraderie of the others involved, makes for a fun creative exercise and certainly helps with motivation.

When I heard about #iDevBlogADay on Twitter, I had to know more about it. It apparently all started when independent iPhone developer @MysteryCoconut wanted the impetus to blog more often, a sentiment that I can definitely relate to myself. What began as an offhand tweet has ballooned into what could become a bonafide movement.

Here’s how it works: Each day of the week is assigned to two indie iOS developers. They must post a blog post on their assigned day. If they miss a day, they’re out and sent to the end of the current waiting list. The next blogger in the waiting list now takes that person’s place.

What’s cool about this is that this kind of motivation helps everyone. The explosion of shared knowledge and inspiration pouring forth from these blogs has been pretty awesome.

It’s now my turn to take a spot on the Sunday roster. I have to admit I’m intimidated, as the quality of the blog posts has been high! I can only hope I can live up to the standards that have been set. (And I apologize for the “fluffy” nature of this post—I had a “crunchier” blog idea that I had apparently been sitting on for far too long, since it was rendered obsolete by recent versions of the iPhone OS. That’ll teach me!)

Hats off to MysteryCoconut for starting a fun “game” that helps and fosters the entire iDevelopment community :)

Core Graphics image masks are handy, but if you want to load the mask image from a file, things don’t always work the way you expect.

The function CGImageCreateWithMask() can take either a mask or an image as the second parameter, but it turns out that Core Graphics (at least on iOS) is pretty picky about what is an acceptable image for the mask.

I’ve seen this snippet of code suggested in a few places:

CGImageRef mask = CGImageMaskCreate(CGImageGetWidth(image),
CGImageGetHeight(image), CGImageGetBitsPerComponent(image),
CGImageGetBitsPerPixel(image), CGImageGetBytesPerRow(image),
CGImageGetDataProvider(image), NULL, false);

the idea being that you create a mask with the pixels that are in the loaded image, but it turns out that this code is not 100% reliable either.

The truth of the matter is that CGImage is an incredibly versatile object. The bits that represent the image can be in a variety of formats, bit depths, and colour space. When you load an image from a file, you are not guaranteed what format those bits are going to be in—for example, there are reports online of how people can get image masks to work if they save it in one way from an image editing program, but not if they save it a different way (e.g. http://stackoverflow.com/questions/1133248/any-idea-why-this-image-masking-code-does-not-work )

Thus, I’ve found that the best and most reliable way to generate an image mask from an arbitrary image is to do this:

1. Create a bitmap graphics context that is in an acceptable format for image masks
2. Draw your image into this bitmap graphics context
3. Create the image mask from the bits of the bitmap graphics context.

The following function has worked well for me so far:

CGImageRef createMaskWithImage(CGImageRef image)
{
    int maskWidth               = CGImageGetWidth(image);
    int maskHeight              = CGImageGetHeight(image);
    //  round bytesPerRow to the nearest 16 bytes, for performance's sake
    int bytesPerRow             = (maskWidth + 15) & 0xfffffff0;
    int bufferSize              = bytesPerRow * maskHeight;
 
    //  we use CFData instead of malloc(), because the memory has to stick around
    //  for the lifetime of the mask. if we used malloc(), we'd have to
    //  tell the CGDataProvider how to dispose of the memory when done. using
    //  CFData is just easier and cleaner.
 
    CFMutableDataRef dataBuffer = CFDataCreateMutable(kCFAllocatorDefault, 0);
    CFDataSetLength(dataBuffer, bufferSize);
 
    //  the data will be 8 bits per pixel, no alpha
    CGColorSpaceRef colourSpace = CGColorSpaceCreateDeviceGray();
    CGContextRef ctx            = CGBitmapContextCreate(CFDataGetMutableBytePtr(dataBuffer),
                                                        maskWidth, maskHeight,
                                                        8, bytesPerRow, colourSpace, kCGImageAlphaNone);
    //  drawing into this context will draw into the dataBuffer.
    CGContextDrawImage(ctx, CGRectMake(0, 0, maskWidth, maskHeight), image);
    CGContextRelease(ctx);
 
    //  now make a mask from the data.
    CGDataProviderRef dataProvider  = CGDataProviderCreateWithCFData(dataBuffer);
    CGImageRef mask                 = CGImageMaskCreate(maskWidth, maskHeight, 8, 8, bytesPerRow,
                                                        dataProvider, NULL, FALSE);
 
    CGDataProviderRelease(dataProvider);
    CGColorSpaceRelease(colourSpace);
    CFRelease(dataBuffer);
 
    return mask;
}

Example of use:

UIImage *maskSource = [UIImage imageNamed:@"mask.png"];
CGImageRef mask = createMaskWithImage(maskSource.CGImage);

Then use the mask as you wish, for example in the aforementioned CGImageCreateWithMask() or CGContextClipToMask()

And don’t forget to dispose of the mask when you’re done. createMaskWithImage() returns the mask with a retain count of 1, and expects the caller to take ownership.

CGImageRelease(mask);

I spent this evening updating my “iPhone VR” Javascript/CSS demo to work with iPhone OS 3.0 (it had stopped working 100% correctly since the OS update). I also decided to spend some time making it work as an offline-capable web application.

The basic process of making your web app cacheable offline is, in theory, fairly straightforward, and generally well-documented at Apple’s website. I ran into some interesting headaches though.

Gotchas:

• First thing to note is that your web server must serve your cache manifest file with a MIME type of text/cache-manifest. This may mean editing mime.types to add a line that looks like this:

  text/cache-manifest manifest

  text/cache-manifest manifest

  or perhaps a line in httpd.conf that looks like this:

  AddType text/cache-manifest .manifest

  AddType text/cache-manifest .manifest

• The next thing that had me stumped for a while: while your web server may know how to take the URL http://example.com/webapp/ and automatically and invisibly serve up the file http://example.com/webapp/index.html, your offline web app knows nothing of this mapping of webapp/ to webapp/index.html. If the URL bar reads http://example.com/webapp/ and you save it locally using a home screen bookmark, it will fail to launch correctly if the device is offline, even if index.html file has been cached. The device simply does not know to look for webapp/index.html instead of webapp/. Thus, you must ensure that the URL bar reads webapp/index.html before the user makes a home screen bookmark.
• At first, I thought I would enforce the above with a tiny bit of JavaScript that simply reads window.location and redirects to window.location + "index.html" if the URL ends in a slash. This worked while online, but it broke my app when offline, even when the redirection was not taken. Why? It seems that any reference to window.location in your script is treated as network access. Since the device is offline, it generates an error alert.

  Instead, I made my app live at webapp/main.html, and created a small webapp/index.html file that simply redirects to main.html. (I could have used a server redirect inside of an .htaccess file instead, but I chose not to, for no particular reason.)

Tips:

• You can specify a custom icon for the home screen bookmark using a <link> element, like so:

  <link rel="apple-touch-icon" href="custom_icon.png" />

  <link rel="apple-touch-icon" href="custom_icon.png" />

  If you don’t want iPhone to automatically apply the “shine” effect on your icon, use the following instead:

  <link rel="apple-touch-icon-precomposed" href="custom_icon.png" />

  <link rel="apple-touch-icon-precomposed" href="custom_icon.png" />

• Since iPhone OS 3.0, offline web apps can have custom splash screens, just a like a native app! Simply add a link element to your web page:

  <link rel="apple-touch-startup-image" href="/splash.png" />

  <link rel="apple-touch-startup-image" href="/splash.png" />

If you want to see the results of all this, go to http://bunnyherolabs.com/iphone/xform/ on your iPhone or iPod Touch, then click the “+” (add bookmark) button and choose “Add to Home Screen.” Now you can click on the “Panorama” icon on your home screen to see the demo at any time, even when not connected to the internet.

Yesterday, when I logged into the iPhone Dev Centre, I was greeted with a “please agree to the new developer agreement” alert. As usual, I copied the text and diffed it with the previous version. The bulk of the changes had to do with allowing in-app purchases in free apps.

But the following change caught my eye: in section 12.2, “Termination”:

12.2 Termination
This Agreement and all rights and licenses granted by Apple hereunder and any services provided hereunder will terminate, effective immediately upon notice from Apple:
…
(f) if You engage, or encourage others to engage, in any misleading, fraudulent, improper, unlawful or dishonest act relating to this Agreement, including, but not limited to, misrepresenting the nature of Your submitted Application (e.g., hiding or trying to hide functionality from Apple’s review).

(emphasis added)

So be careful, hidden-unlockable-feature-mongers. Apple could terminate your licence!

Verizon’s hard-hitting “Droid Does” anti-iPhone ad (via TechCrunch):

I love my iPhone (and developing for it) but I definitely would like to see more competition in this space. I think this ad hits home for those of us who are frustrated with the iPhone’s shortcomings. On the other hand, the style of the end of the video makes me think of trailers for horror movies. I’m not sure that’s such a good association to make!

This ad campaign also reminds me of the old Sega Genesis commercials back in the day:

It’s been a few days now since Adobe announced that Flash CS5 Professional will have a “publish for iPhone” feature for ActionScript 3 projects. Just wanted to jot down a few thoughts:

As a Flash developer and geek, the technology seems pretty damn impressive to me. It actually includes the LLVM compiler? Wild.

Still, there are technical concerns, although to be fair, there are many months before CS5 ships. And for the moment, Flash-built apps won’t have access to things like the iPhone’s native UIKit controls, but they will have access to the accelerometer and multitouch (which at first I thought they did not).

Furthermore, as an iPhone developer, I have concerns, and in a way these concerns have less to do with Adobe’s actions than Apple’s: the single chokepoint that is the App Store and its review/approval system. It’s clear that Apple’s review system does not scale (longer and longer delays in approvals), and discoverability is bad enough as it is with the number of apps in the store now and the limited number of ways there are to browse and find things in the store.

If the iPhone app ecosystem was completely open, with many “stores” and multiple ways of finding and buying apps, I’d welcome Flash-built iPhone apps with open arms: the more the merrier. As it is, though, I worry a bit about the flood of muck as every Flash developer (over a million by Adobe’s count: A MILLION!) with a back catalogue of content tries to get their old code into the App Store.

Some obvious predictions:

• We’ll see more than a few Flash component libraries that emulate UIKit controls
• Apple will unofficially delay or reject Flash-built apps for the first while until/unless Adobe and Apple come to some kind of understanding (see the issues that PhoneGap apps have had in the past, and that uses all native SDKs!).
• Flash developers will find it more difficult than they expect to get their old code working well on the iPhone
• Many iPhone programming contracts will be lost as clients decide (correctly or incorrectly) that they can do their iPhone project in-house with Flash

Still, I have to admit I personally can’t wait to get my hands on the public beta of Flash CS5. I enjoy working with Flash and ActionScript.

Or actually with UITabBarControllerDelegate, and specifically, with the method tabBarController:didSelectViewController:.

According to the docs, the differences:

If you are writing code that runs on both 2.x and 3.0 that needs to get tricky with tabs, these changes are a nuisance.

Even more infuriating is that the Apple documentation isn’t complete. The section ends with the sentence fragment “If you are implementing….” Yes? If I am implementing what? And what do I do if I am?

Yes, I’ve already reported documentation error. No response though. My guess is they’ll just remove the sentence fragment instead of expanding on it :(

[edited: I made a mistake in the table the first time I posted this. Should be fixed now.]