Hamish Rickerby

Technology Consultant & iOS Developer based in Sydney, Australia

Updating a UITableView Without Calling reloadData

| Comments

For a new app I’ve been working on for the past few days I wanted to a nice “inline” way of gathering data from the user. Typically, if I’ve needed to have a user put in more than a single line of text, I would have popped the user to a different screen that has a UITextView as the sole point of focus, get them to type in the data, and then have them navigate back and have the data appear within a tableview.

For this new app, I wanted them to be able to enter large amounts of text inline within a cell of a tableview. This also means that the cell would have to dynamically grow and shrink, as the user is entering the data. The UITextView (where the user is entering the data) would need to resize itself based on the users input, as well as the UITableViewCell that contains the text view, and have the UITableView adjust on the fly to the users input.

What I needed to happen was for the UITableView to go through the process of querying the height of each of the cells via the UITableViewDelegate’s tableView:heightForRowAtIndexPath: method, and executing the necessary layout code to push/pull the cells around as an individual cell expands or shrinks. The other complication is that the UITextView that the user is typing in cannot lose focus (i.e., it cannot resignFirstResponder). The obvious way to get the UITableView to perform layout is to call [tableview reloadData], however, this causes the UITextView to lose focus, and the keyboard disappears. This seems to happen when creating the cells via the tableview:cellForRow:atIndexPath: method.

Then I found a sneaky trick.

If you execute the following code the table view will query the height for the individual cells, and lay them out, but not reload the cells, and not cause the UITextView to resignFirstResponder.

[tableview beginUpdates];
[tableview endUpdates];

So, now I have cells that can grow and shrink dynamically, and not lose focus for the user as they are inputting data.

Core Data Migrations and Large Data Sets

| Comments

I recently updated Moving Van (you should buy it now!) and published the new version in the app store late last week. It was a substantial update to the application - it had a completely new UI with custom interface controls, as well as a whole stack of features that customers had been asking for - things like room autocompletion, saving images to camera roll, more export options, moving items between boxes etc.

As part of this update, I also remodelled the Core Data entities that power the application. The initial model that was used was, let’s say, a little naïve in terms of the way that the stored data would impact performance of the application. It stored an image on an item as binary data within the Item entity itself, which in retrospect was a terrible idea because of table view performance. The new version split out the image to a separate entity, which means that when the Item entity loads, the image doesn’t get loaded unless explicitly needed because of the faulting behaviour of Core Data and entity relationships.

So, to get out of this historic design decision, a data migration was required. The migration itself was relatively simple, with pretty much everything working from a standard mapping model (add two entities, copy existing entities, create relationships with new entities). I had to use a custom migration policy for one aspect of the migration - two image entities are created for each item (for tableview performance reasons). There is the original image, and a thumbnail version of that image. The custom policy needed to take the original image from the source Item, scale the image down, and set it in the new Thumbnail entity, but that itself was relatively simple.

The migration was tested with all possible permutations of the data that a user could create, including a large data set with over 100 boxes and hundreds of items. The migration would take a few seconds to run, and everything was working well. I submitted, and released the new version.

DISASTER.

It appears that my data sets for testing were inadequate. Quite a few users of the application store images for every one of their items. 300, 400 of them. Some users don’t even use the text descriptions for items, they just use images. The larger data sets used for testing were text only - none of the testing involved hundreds of items with images. A database with around 500 images is about 300Mb - I think that’s quite a large CD store for the iPhone.

What was happening is that Core Data, while doing the migration, was choking trying to load all the Item entities (with images embedded) into memory. The lightweight migration mechanism seems to try to be fast, over being resource efficient. On the iPhone this is a bad thing if you have a large volume of data - your application will be terminated with little to no warning.

Apple have specific recommendations for what do with large core data sets - mainly around splitting a lightweight migration into separate mapping models. This approach is fine if you have a large number of entities, but it a useless strategy if you have a large number (or more precisely, a large data volume) of an individual entity. Their “chunks” of data refer to a per entity chunk - the approach still attempts to load all instances of an entity into memory. What I needed was a way to have multiple “chunks” of a specific entity, so the whole set was not loaded into memory at once.

The approach I took to solve this problem is very “manual”. It consists of the following steps:

  1. Determine if a migration is required - if so, pop a migration controller that informs the user a migration is taking place, and start the migration.
  2. Create a Core Data stack with the “old” model, and old version of the store as a source.
  3. Create another Core Data stack with the “new” model, and a new store as the destination.
  4. Request a set of entities from the old data store, with a small batch size to avoid loading all entities at once.
  5. Traverse the object graph of those old entities, creating each instance of an entity in the new data store.
  6. Save the new store every 10 or so entities - this is to ensure that the NSManagedContext doesn’t consume too much memory with unsaved objects hanging around.
  7. After this is all finished, backup the original data store, and move the new one to take its place.
  8. Finally, post a notification for the AppDelegate to receive, that signals the migration is complete and the rest of the startup sequence can continue.

The approach works - the application no longer runs out of memory on migration. However, the mapping model is now useless as it’s never used, and there are a couple of interesting points. First one is that the migration takes up extra storage space as we are creating an extra store with pretty much the same volume of data in it - just laid out differently. I’m not sure if this happens when CD performs a migration - I suspect it is, but what worries me is that if a user is low on space, the migration could cause the disk to fill up. The other thing that I noticed was that the migration is considerably slower that a CD managed lightweight migration. However, it actually works on large data sets, unlike the CD managed lightweight migration, so the positives here outweigh the negatives.

There is probably a way to solve this that utilises more of the Migration classes that Apple provide - specifically subclassing NSMigrationManager - but, I didn’t really have enough time available to figure that out - I needed a fix now.

And now some code.

Determining if your Core Data store needs to be migrated

// See if a database exists to be migrated
NSString *sourceStorePath = <Your source store path in the file system>
if (![[NSFileManager defaultManager] fileExistsAtPath:sourceStorePath]) {
  // Database doesn't yet exist. No need to test data compatibility"
  return NO;
}

// Create a persistence controller that uses the model you've defined as the "current" model
NSURL *modelURL = [[NSBundle mainBundle] URLForResource:@"<Your models directory name>" withExtension:@"momd"];
NSManagedObjectModel *model = [[NSManagedObjectModel alloc] initWithContentsOfURL:modelURL];
NSPersistentStoreCoordinator *psc = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:model];

NSError *error = nil;
NSURL *sourceStoreURL = [NSURL fileURLWithPath:sourceStorePath];
NSDictionary *sourceStoreMetadata = [NSPersistentStoreCoordinator metadataForPersistentStoreOfType:NSSQLiteStoreType
                                                                                               URL:sourceStoreURL
                                                                                             error:&error];
// Do error checking... Removed from the code sample.
NSManagedObjectModel *destinationModel = [psc managedObjectModel];
BOOL pscCompatible = [destinationModel isConfiguration:nil
                           compatibleWithStoreMetadata:sourceStoreMetadata];
// if pscCompatible == YES, then you don't need to do a migration.

Loading old and new Core Data Stacks

You’ll need to do this twice - just swap out the model name for old/new models and keep the references to the MOCs that are created. Ensure you have a different store path for your new store! For the new model, it’s a good idea to also test if a file exists at the new model location - it could be indicative of a migration that’s previously failed. NSURL modelURL = [[NSBundle mainBundle] URLForResource:@“” withExtension:@“mom” subdirectory:@“.momd”]; NSManagedObjectModel model = [[NSManagedObjectModel alloc] initWithContentsOfURL:modelURL]; NSPersistentStoreCoordinator *psc = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:model];

// Get the store url
NSString *sourceStorePath = <Your source/destination store path in the file system>
NSURL *sourceStoreURL = [NSURL fileURLWithPath:sourceStorePath];

// Use this for source store - ensures you don't accidentally write to the entities
NSDictionary *options = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:1]
                                                    forKey:NSReadOnlyPersistentStoreOption];

// Use this for destination store - makes it writeable
NSDictionary *options = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:0]
                                                    forKey:NSReadOnlyPersistentStoreOption];
NSError *error = nil;
[psc addPersistentStoreWithType:NSSQLiteStoreType
                  configuration:nil
                            URL:sourceStoreURL
                        options:options
                          error:&error];
// Do error checking... Removed from the code sample.
NSManagedObjectContext *moc = [[NSManagedObjectContext alloc] init];
[moc setPersistentStoreCoordinator:psc];
[moc setUndoManager:nil];

Get your entities from your original store, and create them in the new store

You can’t use your entity classes here, everything has to be done via KVC. This is because your entity classes will no longer map to the old model correctly. NSFetchRequest oldFetchRequest = [[NSFetchRequest alloc] init]; NSEntityDescription oldEntity = [NSEntityDescription entityForName:@“EntityName” inManagedObjectContext:oldContext]; [oldFetchRequest setEntity:oldEntity]; // Set the batch size so we don’t attempt to retrieve all the data at once - this is the key to the whole thing! [oldFetchRequest setFetchBatchSize:10];

NSError *error = nil;
NSArray *entities = [oldContext executeFetchRequest:oldFetchRequest error:&error];
int count = 0;
for (NSManagedObject *oldEntity in entities) {
  // Creating new entity
  NSManagedObject *newEntity = [NSEntityDescription insertNewObjectForEntityForName:@"EntityName"
                                                             inManagedObjectContext:newContext];
  [newEntity setValue:[oldEntity valueForKey:@"someAttribute"] forKey:@"someAttribute"];

  // If your entity has relationships...
  for (NSManagedObject *aRelatedEntity in [oldEntity mutableSetValueForKey:@"someRelationship"]) {
    NSManagedObject *newRelatedEntity = [NSEntityDescription insertNewObjectForEntityForName:@"RelatedEntityName"
                                                                      inManagedObjectContext:newContext];
    [newRelatedEntity setValue:[aRelatedEntity valueForKey:@"someOtherAttribute"] forKey:@"someOtherAttribute"];
  }
  // Save periodically
  count++;
  if (count % 10 == 0) {
    [newContext save:&error];
    // Do some error handling
  }
}
[newContext save:&error];
// Do some error handling
// Migration is complete, if you've traversed all your entities.

When I encountered this problem I couldn’t find any example code for how to do this migration - hopefully this helps someone.

If anyone does know of an alternative (better) way to get around this issue, please let me know in the comments.

UISearchBar’s UITextField

| Comments

For an update that I’m making to Moving Van (you should buy it now!) I need to customise the font that is displayed in a UISearchBar’s text field. The search bar does not actually expose it’s UITextField property, but because the search bar is a UIView, it’s trivial to access the field to allow customisation.

for (UIView *searchSubview in mySearchBar.subviews) {
  if ([searchSubview isKindOfClass:[UITextField class]]) {
    // Do your text field customisation in here
    [(UITextField *)searchSubview setTextColor:[UIColor redColor]];
  }
}

HTH.

iBooks Crashing?

| Comments

I’m having a bit of a play with iBooks today, and some 3D model stuff. iBooks has, for me, always crashed when displaying 3D models. I don’t have a jailbroken iPad, so it’s not a DRM issue (at least not one caused by that). To tell you the truth, I’m not sure what the cause is. But, I do have a fix!

  • Delete iBooks
  • Reinstall iBooks

That simple. My books and PDF’s were still on the iPad, so there was no issue with having to resyc everything. Now, 3D models are supported :-)

Housekeeping

| Comments

I currently have around thirteen or so applications developed for iOS that are live in the App Store (and one in development). I have one Mac application (and one in development). I also run an online accounting system, two T-Shirt stores, a postcard printing and shipping business, and a web hosting and technical website build business.

That’s a lot of stuff for one guy to maintain. Because of the amount of time I have available to dedicate to each, it can be a long time before I get to maintain or market any particular application or service that I’ve developed. Because of this, the services don’t change often, they underperform in the marketplace, and over time the revenues have been decreasing.

With the move to Melbourne, we (my wife and I) decided that I’d be concentrating more on what I really enjoy doing. There are essentially two things that fit into this category.

  1. Developing iOS and Mac applications for myself, and others.
  2. Advising and consulting with people on their technical business ideas.

To be able to concentrate on this, I’ve decided that it’s time for some housekeeping. I’ll be shutting the doors on some of the applications and services that I maintain. We’ll be saying bye to:

  1. Sendit4.me - iOS application, website and facebook application for sending physical postcards. Apple sherlocked me with their Cards app. Since this launched, I’ve had virtually no sales via the iOS app, and this was the main sales channel.
  2. My 2 T-Shirt stores - they cost virtually no money to run, but suffer from a lack of marketing resulting in poor sales. Online T-shirt stores is also a very competitive space.
  3. Got the GiST (GST accounting package for small businesses in New Zealand) - this is the first application I ever published by myself.
  4. I’ll be looking to do less of the web site development work. It’s easy work, the money is OK, but I just don’t really enjoy it.

I’ll be concentrating more on:

  1. The remaining iOS applications that I have that are currently doing OK commercially - these are the CBT applications (Mental Health), Photo Copy, Moving Van, and Gifted TY.
  2. Crate Digger - this is a labour of love - I use the application myself, and it does need an update. It has good reviews, and although it makes no money, I like developing it - it’s a hobby app.
  3. Marketing my own application consulting, design and development services to other businesses - looking to grow that segment of the work that I do.

The other applications I’ve released will remain as is. It’s unlikely there will be any updates to them, and when they break with a new iOS release, I’ll probably pull them from sale.

With the decreased maintenance overhead, I’ll hopefully have more time to dedicate to updating and marketing the applications I’ve developed. I need to curate a portfolio of quality applications.

If anyone is interested in purchasing the code for any of the applications that are being retired or no longer updated just let me know.

Unit Test Code Coverage With Xcode 4.3.2

| Comments

I upgraded to Xcode 4.3.2 and this seemed to break unit tests on iOS for me.

The error that was being reported was: Test rig ‘/Applications/Xcode.app/…/iPhoneSimulator.platform/…/otest’ exited abnormally with code 134 (it may have crashed).

(… used to save space)

The actual cause of this error was explained in the unit test build/run log files in Xcode: Detected an attempt to call a symbol in system libraries that is not present on the iPhone: fopen$UNIX2003 called from function llvm_gcda_start_file in image UnitTests.

I covered this issue in a prior post, but the short version of how to get around it is to include the below code in one of your test .m files, outside of the @implementation…@end block. I recommend right down the bottom of one of the files.

#include <stdio.h>
// Prototype declarations
FILE *fopen$UNIX2003( const char *filename, const char *mode );
size_t fwrite$UNIX2003( const void *a, size_t b, size_t c, FILE *d );

FILE *fopen$UNIX2003( const char *filename, const char *mode ) {
  return fopen(filename, mode);
}
size_t fwrite$UNIX2003( const void *a, size_t b, size_t c, FILE *d ) {
  return fwrite(a, b, c, d);
}

Other settings for successful execution of the unit tests with code coverage are (in your Project’s Unit Test target):

  • Generate Test Coverage Files = YES
  • Instrument Program Flows = YES

There is no need for linking libprofile_rt to get coverage to work.

Continuous Integration and iOS

| Comments

The client project I’m currently working on is quite large. There are over 90 different screens required in the application, and regression testing all of these, with the different data variants and scenarios is not something I’d like to attempt by hand, and is not something I would expect my client to pay for me (or anyone for that matter) to do. Both unit and application tests can be automated, and I figured that this was something that I should do for this project to ensure changes I introduce don’t break existing functionality, and any bugs found won’t be introduced.

To solve this challenge I have introduced a Continuous Integration (CI) server to my workflow. My goals were to have unit and application tests automatically executed when an commit is made to a specific branch in a (local) git repository, and if both of these phases are successful, have the application packaged up, ready for distribution. I also wanted this successful build to be deployed on a daily basis to TestFlight so my client (and any other human testers in the future) could pick it up on their devices. I also wanted code coverage reporting, for both the unit and application tests, so I can see what parts of the application logic and screen flows are actually being tested to give me a level of confidence that the right stuff is getting the attention. The application tests also needed to be executed in a headless manner - I don’t have a physically separate machine (or VM) to run the CI server in, so I don’t want the simulator popping up and distracting me.

Warning - this is long and involved. I’ve also written this after the fact. I hope I’ve captured all the steps but if something doesn’t work for you please let me know and I’ll try to help and update this guide.

Toolkit

I used a set of existing tools to help with this

  1. CI Server - Jenkins
  2. Unit Tests - SenTest/OCUnit - it’s baked right into Xcode and meets my needs. It’s hard for me to justify using something else like GHUnit because of the pre-integration.
  3. Application (UI) Tests - Frank - I wanted to use a behaviour driven approach to UI testing, as well as something my client could actually specify tests in. Frank is cucumber based, so uses an english language syntax, making it easy for non-developers to specify and understand tests.
  4. Deployment - Curl - low tech, but the TestFlight API is simple to use, so nothing more complex is really required here. It could be wrapped in a Jenkins plugin, but I’m not the guy to create that…

And this is how I did it. I cobbled together all the tools I needed with help from various blogs, stack overflow answers, and vendor documentation. I’ve referenced the sources where I can. Hat tips to all.

Installation of Prerequisites

Install rvm

You’ll need ruby for compiling Frank, and I recommend you use rvm for this. Paste the following into a shell and follow the instructions.

bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)

Extra install instructions (head builds, multi-user etc) are available at http://beginrescueend.com/rvm/install/

You’ll need to close and open shell after following the instructions (including modification of your .bash_profile). You’ll also need to install your favourite ruby. I recommend installing 1.9.2 with rvm install 1.9.2

You’ll also need rake (for managing Frank’s build process), so you can get that with gem install rake

Install homebrew

I used homebrew to install Jenkins. It’s a good package manager for OSX, and well maintained. Instructions are at https://github.com/mxcl/homebrew/wiki/installation but you can just paste the following into a shell and it’ll install.

/usr/bin/ruby -e "$(/usr/bin/curl -fsSL https://raw.github.com/mxcl/homebrew/master/Library/Contributions/install_homebrew.rb)"

Install Python and gcovr

gcovr (and python) are needed to generate the coverage files from the unit and application test output. If you already have a modern python installation (2.7) then skip the python installation step and just install gcovr with your own copy of easy_install

You can use homebrew to install python. Do so with:

brew install python

Then you will need easy_install (or the distribute tools)

1
2
curl -O http://peak.telecommunity.com/dist/ez_setup.py
/usr/local/bin/python ez_setup.py

You will then need to get gcovr.

/usr/local/share/python/easy_install gcovr

Install and configure Jenkins

Thanks to homebrew, the installation step is very easy. Just paste the following into the shell and it’ll install.

brew install jenkins

There are a couple of configuration steps that are required as well. Thanks to http://mattonrails.wordpress.com/2011/06/08/jenkins-homebrew-mac-daemo/ for the configuration required.

Create a service account

First of all find an ID that is free on your system. dscl . -search /Users uid 600 searches for users with ID 600, and dscl . -search /Groups gid 600 looks for groups with ID 600. Change the number until you find an empty ID. Then (with appropriate ID changes)…

1
2
3
4
5
6
7
8
9
10
11
12
13
sudo mkdir /var/jenkins
sudo /usr/sbin/dseditgroup -o create -r 'Jenkins CI Group' -i 600 _jenkins
sudo dscl . -append /Groups/_jenkins passwd "*"
sudo dscl . -create /Users/_jenkins
sudo dscl . -append /Users/_jenkins RecordName jenkins
sudo dscl . -append /Users/_jenkins RealName "Jenkins CI Server"
sudo dscl . -append /Users/_jenkins uid 600
sudo dscl . -append /Users/_jenkins gid 600
sudo dscl . -append /Users/_jenkins shell /usr/bin/false
sudo dscl . -append /Users/_jenkins home /var/jenkins
sudo dscl . -append /Users/_jenkins passwd "*"
sudo dscl . -append /Groups/_jenkins GroupMembership _jenkins
sudo chown -R jenkins /var/jenkins

Starting up Jenkins

Create a file at /Library/LaunchDaemons/org.jenkins-ci.plist with the following contents (check the version number in directory for Jenkins!). This will ensure Jenkins starts when the system boots.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>Jenkins</string>
    <key>ProgramArguments</key>
    <array>
    <string>/usr/bin/java</string>
    <string>-jar</string>
    <string>/usr/local/Cellar/jenkins/1.428/lib/jenkins.war</string>
    </array>
    <key>OnDemand</key>
    <false/>
    <key>RunAtLoad</key>
    <true/>
    <key>UserName</key>
    <string>jenkins</string>
</dict>
</plist>

and load it with sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

At this point, Jenkins should be installed and running. Head to http://localhost:8080 and it should be running.

Configuring Jenkins

In the Jenkins menu, click on Manage Jenkins, Manage Plugins, and then Available. Install the following plugins and make sure the “Restart Jenkins when installation is complete and no jobs are running” checkbox (on the screen after the “Install” button) is checked.

  • Jenkins Cobertura Plugin - for code coverage report processing
  • Jenkins GIT plugin - for integration with git
  • Xcode Integration - for execution and understanding of Xcode files

Install the github plugins if your project is on github. I won’t be covering configuration of this here, but it should be relatively simple.

Once these are installed and Jenkins has restarted, you’ll need to configure the plugins. In the Jenkins menu, click on Manage Jenkins, Configure System and fill in the parameters for your git installation(s) and Xcode Builder paths. Click Save and we’re ready to go.

Configure your Xcode project

We’re going to take a little departure from Jenkins for the moment to prep Xcode for integration. We are going to setup our project for unit tests and Frank.

Unit Tests

If you don’t have one already, you’ll need to set up a new target for Unit Tests in your Xcode project. Select your top level project in the in Project Navigator, and then “Add Target”. Under iOS -> Other choose the “Cocoa Touch Unit Testing Bundle”. Give it a name and then write some tests.

The other thing we’ll need to do here is set up Code Coverage. In the Build Phases area of your new Target, under Link Binary With Libraries, hit the + and select Add Other. Then navigate to /Developer/usr/lib and select libprofile_rt.dylib. This is the library that enables the profiling goodness. After this, select the Build Settings area, and set “Generate Test Coverage Files” and “Instrument Program Flow” both to Yes in the column for your Unit Test target. Ensure that “Library Search Paths” includes $(DEVELOPER_DIR)/usr/lib, but this should be there already.

You should be set up for code coverage now. If you want to check this is working, ensure your build target (up the top on the right on the Run & Stop buttons) is set to your Unit Test target and iPhone Simulator, then build and test your unit test target. Then, open Organizer, choose the Projects item from the toolbar, select your project, and then click the little arrow next to the Derived Data directory. This will open the build location in Finder. In here, open the selected directory, then navigate to Build/Intermediates/<Your Project>.build/Debug-iphonesimulator/<Your Unit Test Target>.build/Objects-normal/i386/ and in there there should be a set of .gcno (generated at build) and .gcda (generated when your test target executed and finished) files. These are the code coverage files. If you’d like to have a look at them before we integrate back into Jenkins, get CoverStory and open them up.

Application (UI) Tests with Frank

Now we’ll set up the application test target with Frank. First of all, we need to install Frank and cucumber.

We will need to use a customised version of Frank to enable the iOS Simulator to exit after execution of the tests. The standard build does not include a method to terminate an application so we’ll need to build this in. Thanks to Martin Hauner at http://softnoise.wordpress.com/2010/11/14/ios-running-cucumberfrank-with-code-coverage-in-hudson/ for the tip on this. I’ve forked frank and included this exit method, and you can get it from https://github.com/rickerbh/Frank/tree/exitCommand with the command git clone [email protected]:rickerbh/Frank.git and then switch to the exitCommand branch. You’ll also need to git submodule init and git submodule update, and then check the submodules that are pulled in as I recall the submodules have submodules :-/

Once all that is done, you’ll need to be in the root directory of Frank that you cloned, and compile my branch of Frank with the following command.

rake build_lib

After this is finished, there should be a file at dist/libFrank.a. This is the customised library that we’ll need to use with the exit command built in.

(Full Frank installation instructions available at http://www.testingwithfrank.com/installing.html - I’ll paraphrase here with a couple of sightly different steps for the custom library and code coverage inclusion) For convenience of installation, I actually installed the proper Frank gem rather than my customised build. You can do this with gem install frank-cucumber. Then, cd to your project directory, and run frank-skeleton. This installs Frank in your project directory. It also copies a version of the official libFrank.a file into Frank/. You’ll need to replace that with the version that we built.

To add Frank to your Xcode project, you’ll need a new target (you don’t want the Frank server installed in the Release version of your application). Duplicate your main application target by right clicking on it and selecting Duplicate. Rename the new target “\ Frankified”. Then, add the Frank directory (that was created when you ran frank-skeleton) to your Xcode project. Ensure that it’s only added to your frankified target, not your main application target. Add CFNetwork.framework to the Frankified “Link Binary With Libraries” section of the Build Phases. Then, add -all_load and -ObjC to the “Other Linker Flags” Build Setting.

To enable code coverage for Frank, add --coverage to the “Other Linker Flags” Build Setting, and set “Generate Test Coverage Files” and “Instrument Program Flow” both to Yes in the column for your Frankified target.

If you build and run the Frankified target for the iPhone simulator of your application, it should build OK and start the simulator. Head to http://localhost:37265 and you should see the Symbiote browser of your iPhone application.

If you want to see if the coverage is working, head to Build/Intermediates/<Your Project>.build/Debug-iphonesimulator/<Your Frankified Target>.build/Objects-normal/i386/ just like with the Unit Tests area above.

You should then write some tests in cucumber and make sure they work.

Troubleshooting Frank and Code Coverage

I had a lot of trial and error (mostly error actually) getting coverage working with Frank. I had to play a bit with the “Library Search Paths” Build Setting so it could find the correct coverage library to include. I have $(Developer)/Platforms/iPhoneOS.platform/Developer/usr/lib added in the setting, but can’t recall if this is required or not - apologies for this.

If you are getting the .gcno created but not the .gcda files, it could be an issue of your application not exiting correctly when the simulator terminates (as the .gcda files are only written when the application terminates). If this is the case, head into the Info area for your Frankified build target. Add/Set the “Application does not run in background” property to the application, and set it to YES. This will ensure the app terminates when the home button is pressed, and the .gcda files are created.

I also had an issue when the application was attempting to write the .gcda files. It was complaining that fopen$UNIX2003 called from function llvm_gcda_start_file. I found this entry in stackoverflow and created the c methods in my main.m file and it enabled the application to write the files correctly.

You may also need to enable the Accessibility inspector in the iPhone Simulators Settings app under General > Accessibility - instructions from Apple.

If you are getting a message that says Couldn't register XXXX with the bootstrap server. Error: unknown error code. This generally means that another instance of this process was already running or is hung in the debugger. this means that either the Simulator is already running (it must be quit for the headless build to run) or that there is a previous version of the headless test exection that has hung. You can find these with ps -ef | grep Fran (Fran for frankified). Kill them with the kill command.

Configuring Frank for Headless Tests under Jenkins

There will be a file in your Xcode project under the Frank directory at support/env.rb - this will contain some environmental settings that are used for executing the cucumber tests. Replace the content with the below with the appropriate text replacements. You will need different settings for manual command line based test execution, and the Jenkins based tests, so there is an environment based conditional in the file. Default is for command line based testing.

For the replacement text for , just type in what you’ll call your Jenkins UI test job, and remember this for later.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
require 'frank-cucumber'

ENV['TESTING_ENV'] ||= 'command_line'
environment = ENV['TESTING_ENV']

if environment == 'command_line'
 BASE_DIR = "<Your home directory>/Library/Developer/Xcode/DerivedData/<Your long and random derived data dir from Xcodes project organizer>/"
 APP_BUNDLE_PATH =  "#{BASE_DIR}Build/Products/Debug-iphonesimulator/<Your frankified target>.app"
 APP_DIR = "#{BASE_DIR}Build/Intermediates/<Your project name>.build/Debug-iphonesimulator/<Your frankified target>.build"
elsif environment == 'jenkins'
 BASE_DIR = "<Your Jenkins install location>/jobs/<Your UI Testing Jenkins Job Name>/workspace/"
 APP_BUNDLE_PATH =  "#{BASE_DIR}build/Debug-iphonesimulator/<Your frankified target>.app"
 APP_DIR = "#{BASE_DIR}build/<Your project name>.build/Debug-iphonesimulator/<Your frankified target>.build"
end

#### Common ####
SDK_DIR = "/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk"
APP_BINARY = "#{APP_BUNDLE_PATH}/<Your frankified target>"
USER_DIR = "iPhone Simulator/User"
PREF_DIR = "#{USER_DIR}/Library/Preferences"

In your Xcode project under the Frank directory, there will be another directory named step_definitions. This should contain a .rb file that has some ruby and cucumber/frank definitions it it. We need to add a couple of things to that file. Again thanks to Martin Hauner here.

Add the following lines to the top of the ruby file in the step_definitions directory. I’ll explain what these are…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
require 'fileutils'

ACCESIBILITY_PLIST   = "com.apple.Accessibility.plist"
ACCESIBILITY_CONTENT = <<PLIST
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>ApplicationAccessibilityEnabled</key>
<true/>
</dict>
</plist>
PLIST

Before do
  # check that pwd contains the "build" dir as we are creating
  # items relative to it.
  #Dir["build"].length.should == 1

  # make sure we do start with a clean environment
  FileUtils.remove_dir("#{USER_DIR}",true)

  pwd     = "#{Dir.pwd}"
  prefdir = "#{PREF_DIR}"
  FileUtils.mkdir_p prefdir

  File.open("#{PREF_DIR}/#{ACCESIBILITY_PLIST}", 'w') do |f|
    f <<ACCESIBILITY_CONTENT
  end

  ENV['SDKROOT']               = "#{SDK_DIR}"
  ENV['DYLD_ROOT_PATH']        = "#{SDK_DIR}"
  ENV['IPHONE_SIMULATOR_ROOT'] = "#{SDK_DIR}"
  ENV['TEMP_FILES_DIR']        = "#{APP_DIR}"
  ENV['CFFIXED_USER_HOME']     = "#{pwd}/#{USER_DIR}"
end

After do
  frankly_exit
end

def launch_app_headless
  @apppid = fork do
    exec(APP_BINARY, "-RegisterForSystemEvents")
  end
  wait_for_frank_to_come_up
end

def frankly_exit
  get_to_uispec_server('exit')
  # calling exit in the app will not return any response
  # so we simply catch the error caused by exiting.
  rescue EOFError
end

Given /^I launch the headless app$/ do
  launch_app_headless
end

The Before block gets executed before any tests are run. The accessibility plist/content section sets up a file that enables the accessibility settings for the simulator. This is created every time the cucumber tests are run to ensure accessibility is in the correct state. The SDKROOT, DYLD_ROOT_PATH, IPHONE_SIMULATOR_ROOT, TEMP_FILES_DIR, and CFFIXED_USER_HOME are all directories for the simulator to function correctly. Thanks to Matt Gallagher’s blog entry for aiding my understanding of these.

The launch_app_headless method adds a flag when launching the application so it launches headlessly. If you wanted, you could actually add a conditional so that the headless method is only used when launching under jenkins.

The frankly_exit method will call the new exit method that we built into libFrank.a. This is called from the After block, which gets called after the tests are executed.

That should be all the configuration you need. If you alter your frank/cucumber tests to use the “Given I launch the headless app” call to start the application, you should now be able to run it in a headless manner. Execute cucumber in your Frank directory inside your Xcode project dir to test it out.

Setting up the jobs in Jenkins

The way I have structured my tasks in jenkins is that I have 4 different jobs. I have a unit test execution job that triggers off a git push. If this is successful, I have the UI test job that executes. If this is successful, I then package and archive the binary that was generated from that push. The last job that is configured is a daily distribution of the last successfully tested application to a group on TestFlight.

Setting up the unit test job in Jenkins

Navigate to Jenkins and click on “New Job”. Give it a name (I called mine “Project Unit Tests”), and select the “Build a free-style software project” radio button, then click OK.

You should now be in a screen to configure the build settings for your unit test target. In the source code management area, choose git. Enter your repo name (something like [email protected]:my-project.git). If you have a specific branch you want to build from put it in the “Branches to build” box (mine is */develop). In the “Build Triggers” area below, select “Poll SCM”. I’ve set my schedule to * * * * * meaning it’ll look every minute for a new push.

Navigate down the screen until you find the “Add build step” button. Click it and select Xcode. Then, fill in the following boxes.

  • Clean before build - check this box. I like clean builds before testing.
  • Target - set this to the unit test target name from Xcode (no escaping of spaces required)
  • SDK - set this to the SDK you want to build for. Mine is /Developer/Platform/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk/
  • Configuration - set this to DEBUG
  • Keychain path - this should already be set to ${HOME}/Library/Keychains/login.keychain

That’s all the Xcode build information set up. Now, to generate the coverage test files. Add an “Execute Shell” build step with the “Add build step” button. In the “Command” box, call gcovr (which gcovr to find your own install location) with /usr/local/share/python/gcovr -r "<Your Jenkins install location>/jobs/Project Unit Tests/workspace" --exclude '.*UnitTests.*' --xml > "<Your Jenkins install location>/jobs/Project Unit Tests/workspace/coverage.xml"

In the Post-build Actions section, check the “Archive the artifacts” box and set the “Files to archive” field to build/Debug-iphoneos/*.ipa

Check the “Publish Cobertura Coverage Report” box and set the “Cobertura xml report pattern” to **/coverage.xml.

Also check the “Publish JUnit test result report” box and set the “Test report XMLs” to test-reports/*.xml.

Click the Save button down the bottom, and then attempt to run your unit tests manually. The code should be checked out from your git repo, build the unit test target, run the tests and produce the unit test reports and coverage results.

Setting up the application test job in Jenkins

Navigate to Jenkins and click on “New Job”. Give it a name (I called mine “Project UI Tests”), and select the radio button to copy the unit test job that was previously set up. In the Configuration screen for the new job, alter the following fields.

  • Build Triggers - set this to be “Build after other projects are built” and type in the name of your unit test job.
  • Build Triggers - uncheck poll scm
  • Target - Set this to your Frankified target name from Xcode

Change the existing “Execute shell” step to use your UI Test build job name rather than the unit test job name. /usr/local/share/python/gcovr -r "<Your Jenkins install location>/jobs/Project UI Tests/workspace" --exclude '.*UnitTests.*' --xml > "<Your Jenkins install location>/jobs/Project UI Tests/workspace/coverage.xml"

Create a new “Execute shell build step”, and click and drag it (with the little 4x4 set of boxes on the right of the “Execute shell” label on screen) to move it inbetween the Xcode step and the gcovr step. Insert the following commands in the shell box.

1
2
3
4
source <Your home dir>/.rvm/environments/<your ruby version>
export TESTING_ENV="jenkins"
cd "<Your Jenkins install location>/jobs/Project UI Tests/workspace/Frank"
cucumber -f junit --out ../test-reports

The source line sets up the environment for rvm so the frank gem is included correctly - set it to the appropriate ruby version in the ~/.rvm/environments directory. Mine is ~/.rvm/environments/ruby-1.9.2-p290

Uncheck the “Archive the artifacts” option, and click save. You can now attempt to run your application tests via Frank. This will check out the code, build the Frankified target, execute the tests and then process the unit test and coverage reports.

FYI - my unit test reports for this step are empty. If someone figures out how to get cucumber to put something useful in them please let me know!

Setting up the archive job

Similar to the application test job, create a new job that copies the Unit Test job. I called mine Project Developer Build. In the Configuration screen for the new job, alter the following fields.

  • Build Triggers - set this to be “Build after other projects are built” and type in the name of your application/UI test job.
  • Build Triggers - uncheck poll scm
  • Target - Set this to your application target name from Xcode (mine is Project TestFlight - I have a specific target that includes the TestFlight SDK)
  • Configuration - I have this set to release. Be sure to setup and test the signing identities correctly in Xcode for this. You must use the adhoc profile that should be used in TestFlight.
  • Build IPA - check this.

Remove the execute shell build step with the Delete button.

Uncheck the Cobertura and Unit test report generation Post-build actions. Change the Archive files location to build/Release-iphoneos/*.ipa

Click save, and now whenever both your unit and application test steps pass, you’ll have a version of the application built and archived ready to go to TestFlight.

Distributing to TestFlight

Thanks to the Shine Technologies team for their blog entry that helped here.

Create another free-style software project job (last one, I promise!) in Jenkins - I’ve called mine Project TestFlight Deployment. Set the build triggers to “Build periodically” and set the schedule to whatever time you want to upload the application to TestFlight. Mine is set to 10pm (0 22 * * *).

Add an Execute shell build step. This will call curl to upload the application to TestFlight. You will need your API token and Team token from Testflight. Set the Command to the following

1
2
cd ../..
curl http://testflightapp.com/api/builds.json -F [email protected]\ TestFlight/lastSuccessful/archive/build/Release-iphoneos/Project\ TestFlight-Release.ipa -F api_token=’<api token>’ -F team_token=’<team token>’ -F notes=’This is an auto deploy build of the develop branch with Release configuration’ -F notify=True -F distribution_lists=’<name of test distribution list>’

The ipa that’s being uploaded there is the version that was saved in the last build job. For the name, just look inside the Project TestFlight job and it’ll have the name of the “Last Successful Artifact” - this is what you need to upload.

That should be it.

End - finally

If you experience issues with this, have corrections or useful troubleshooting steps then please let me know. This process was a bit of a pain to get working correctly so I hope this guide is useful for someone. You should also follow me on twitter @rickerbh.

On hold…temporarily

| Comments

I had some great news last week. A previous client asked me to submit a quote for the full version of a prototype iOS app I had created for him last year. I submitted the proposal, but had assumed he was using it to benchmark other application development companies against. To my surprise, he accepted my proposal, and awarded me the work. It’s a substantial piece of work, and technically quite challenging. It also has the potential to be a big innovation on the space it’s in, so is exciting from that aspect too.

So, I accepted. The issue for me is that this puts my other project on hold for a few months. However, income is good, and this will let me work for longer on my project after I’m done, so it’s a positive move.

User Experience Books

| Comments

As I alluded to in a previous post, things like User Experience, Human Computer Interaction, and Graphic Design are not my fortes. All my professional software experience until I started iOS development was either in developing (~2 years) or designing/architecting (9+ years) large scale telecommunications software systems (specifically BSS and OSS systems if you’re interested). These sorts of systems do not have pretty user interfaces - there is no need to provide them. In fact, the majority of systems I have designed only have human interactions with back-office operators who are Unix system administrators - the main “user” of the system would typically be another system.

The users of my new software will be proper “end-users”. People who do not use computers for a job. People who are unsure, or even wary about computers for fear of breaking them. People who do not know commands like kill -9 when something isn’t working, or even how to drive a CLI.

It would be delinquent of me to not take some advice from well respected professionals in this space. So, I read some books. (I will take some advice from actual people later, but as a first step I think education is always a good step.)

Designing Interactions by Bill Moggridge

Amazon UK, Amazon US

I actually got this book expecting it to have practical advice on how to design software products, and was (pleasantly) surprised when it didn’t. It’s a collection of interviews (and then analysis and commentary on the interviews) with a set of influential product designers. There is a strong bias towards IDEO, but I guess that’s to do with the authors familiarity with the organisation. The book gives insight into the interaction design processes used to create some very popular products, as well as covering some things that haven’t worked. It gives you a good idea of how the designers think about problems - which is great to understand as an outsider to this world.

The two parts of this book that stick with me are Terry Winograd’s discussion on Mark Weiser’s ubiquitous computing - the idea that people don’t want to interact with computers - they want to get something done. The computer is an instrument to be used to get something done - the purpose is not to interact with the computer, but to achieve a goal. The other part was John Maeda’s Laws of Simplicity. These eight laws are about making complex systems simpler for users, either by relating functions, using knowledge that users already have, and recognising when materials (technology in my case) is inappropriate for a task, either by it being too hard to use, or breaking “laws”.

TOG on Interface by Bruce “TOG” Tognazzini

Amazon UK, Amazon US

I got this book because of the Apple Macintosh legacy the author has. Bruce Tognazzini developed the first set of Human Interface Guidelines for Apple in the late 70’s, and was involved with the Macintosh UI which still has a significant influence on the modern interface we use on OSX. The book is a collection of his writing for the Apple Direct publication as well as other text gathered or produced for this book. The book was written in 1992 (updated in 1996 - 15 years ago!) but remains relevant. The principles are as relevant today as they were way back in the 90’s.

Three things made this book useful to me. The first are all Bruce’s Principles and Guidelines (which are outstanding) and the collection of them in the appendix. This gives you an overview (and shortcut to relevant places in the book) for each item he discusses in the book. The second is (and this may sound odd given I’ve used a mac now for 5 years) he describes what the ellipsis means in a Mac menu or on some buttons. I remember seeing it, but I never thought about why it is there. It means that a window or pane will pop up asking for more information before the action you have requested will be executed. This is all about letting users know they can experiment with the system and be assured they can get out again without making any changes. This was a little UI thing I just hadn’t consciously grasped the meaning of, and it’s just nice to understand why it’s there and what it means. The third thing was the in-depth discussion about the Conceptual Model a user generates when they interact with a system. This was like a lightbulb going off for me. This is such a fundamental part of any system and user interface design. It’s obvious it happens (users have a view of what systems do, which may or may not be correct), and designers need to cater for this - exposing elements to users that influence that conceptual model, to make it easier for users to understand and interact with the system. I’ve got a lot of thoughts on this so may write another entry on it.

Anyway, I would definitely recommend both books to anyone involved with software product design - I’m not sure how much new information there will be for experienced UE practitioners, but I’m sure there are some useful nuggets in both for all.

Sandboxing

| Comments

Since the last post and this one, we’ve moved 10500 miles (that’s nearly 17,000 kilometres!) and only yesterday got all our utilities sorted out, meaning that I’ve basically been offline (bar mobile phone for the past 3 weeks). Melbourne is fantastic. I really like it here.

Anyway, I’ve been doing a little bit of work on my application, mostly completion of the requirements, data model, and competitor analysis, as well as reading up a bit on UX and Mac way of doing things (I’ll probably cover this in another entry), and most recently have been prototyping some code that I thought would be particularly tricky.

What this code does is synchronise data across other applications on your mac, with my (intended) application. The reason behind this was to make the act of populating my application with data, and keeping the data fresh over time, simpler for users - or as John Maeda states in his Second Law of Simplicity:

The positive emotional response derived from a simplicity experience has less to do with utility, and more to do with saving time.

I’m into saving time for my (future) happy users, and reusing data that is already on their Mac helps with this - they don’t need to double key or manually join-the-dots between the applications.

The prototyping I was creating was using ScriptingBridge to access data that’s available in other applications that a user may use. The prototype was going well - I had managed to get some queries and filtering down from 5 minutes to 90 seconds to 6 seconds which was a nice level optimisation, and although there are some issues with ScriptingBridge, it was going to let me do what I wanted to do.

One of the key things with the distribution of this application for me is to have it in the Mac App Store. New mac users are a growth market (look at the stats), and Apple is pushing them down the Mac App Store route to discover and purchase applications. Not having the app in the Mac App Store would be foolish from my perspective.

Today, I was seeing how I could get the application working under sandboxing - as this is now a requirement for new Mac Apps. It’s all about making applications safer for users in terms of the resources they can use in terms of network, disk access etc (although some prominent developers aren’t convinced).

I was sad to discover that ScriptingBridge is precisely one of the technologies that is hit hard by the sandboxing. Basically, applications can receive AppleEvents, send AppleEvents to themselves, and respond to AppleEvents they receive (source). The upshot of this is that AppleEvents are pretty useless in a environment of sandboxed applications, as you can only send them to yourself (and no-one else can send them to you).

There may be a way around this - Temporary Exceptions to the AppleEvent sending issue. You can obtain a temporary exception to send AppleEvents to other applications through the com.apple.security.temporary-exception.apple-events key-value pair in your Entitlements file. The main issues I see with this are:

  1. They are likely to attract more attention in the review cycle - this in general is a good thing as it’s important that Apple check what your application is doing to ensure that it’s being a good citizen. However, this is likely to slow down the review process and I suspect guidance on what’s acceptable is not consistently applied.
  2. They are temporary - Apple could disallow/reject them in the app updates, or just remove the function. The clue is in the name, people.
  3. The intent behind the exception - I suspect that the intention behind the temporary exception is to allow pre-existing applications to migrate to a 100% compliant sandboxing world over time, not for new applications. Approval is at risk here, and I don’t want to sink significant effort into getting rock solid sync only to be told it’s not allowed.
  4. The overall future of AppleEvents - this (in my mind) is a clear intention that AppleEvents will not have a strategic future at Apple, and that existing applications may drop support for them as no sandboxing compliant applications can actually use the functions. It seems to me risky to use a technology that seems to have a limited future.

This leaves me with two options. I can either remove the synchronisation functionality from the application, or I can distribute it outside the Mac App Store. My wife actually suggested a third - do both. I think that’s what I’ll do. Mac App Store is first priority, and depending on the markets response, I’ll look at options for data synchronisation.

Still, it’s a massive PITA.