When I was approached by a group of UX designers to develop their pet project, a social network designed to connect instructors and students in the Dance Fitness community, I was reluctant to volunteer my time towards some external project with an inexperienced team, but I was intrigued at the thought of architecting and producing a social network from scratch.
I was curious about the level of complexity involved in programming something of the likes of Facebook, Instagram and Twitter. That and I felt sorry for them, considering that they had already had an app developed by another software team with disastrous results, and I was bored, at the time, and needed something to focus on during my free time. What better way to spend my free time, I thought, than to put to the test all my accumulated experience with software development and programming patterns. I felt that this would be a great opportunity to validate my ideas and test my assumptions about project management and iOS application architecture.
And thus, the Dance Love App adventure started…
Dance Love App is the vision of Pat Lang, a tech professional with a design / product background and a passion for Dance Fitness. Pat Lang has mostly been involved with tech from a UX standpoint, but he has worn many hats before and was involved in various ventures before with mixed results. He had a decent understanding of the prototyping duties, was pleasant to talk to, and, as the rigors of development started kicking in, showed that he was receptive to feedback and able to handle pressure.
Most self respecting developers would cringe at the thought of taking on a project of that scale with a unproven team for no money. I myself often pontificate that one should never accept to work for free. Yet I also have gained tremendously from working on unremunerated side projects on multiple occasions. Be it during my very first year of programming, when I was locked up in my room reading Mike Daley’s “Learning iOS Game Programming” in late 2010 and programming my very own videogame entitled Dark Realms. Or when I tirelessly made a iOS application of my neighbor’s news aggregating website, MapReport, which he didn’t bother to publish despite the fact that it was working and ready.
Those projects can be insanely frustrating, especially during the massive anti-climax that occurs when the development is complete but the release fizzles.
And yet, despite the repeated disappointment, I keep coming back for more because of reasons.
The first problem I was faced with was reconciling the team’s wild projections with reality. They had a bunch of mocks lined up and despite some undeniable design talent, a lot of the flows had to be clarified. My first initiative was to use JIRA to methodically negotiate and define every requirement according to different app modules.
During the course of the exercise, we were able to achieve three things:
- Differentiate between what was feasible and what was not feasible.
- Clarify how the app would actually work (which button would trigger what transition etc).
- Have a centralized repository which would serve as the source of truth throughout the rest of development.
Once we had agreed on a finite set of documented features, it was time for me to get to work. I had several important architectural decisions to make, both from a programming standpoint, and from an operational standpoint.
First stop source control, choice was easy, I set up a private repo in Bitbucket. I knew that I could count on two of my friends to “augment” me if needed (Jon Olivet and Miguel Garcia), and created development tasks from the JIRA stories, which they could use to create branches and submit pull requests for my approval. For traceability I used the JIRA number in the commit message because I always found having the changesets automatically listed under the JIRA ticket, and having the JIRA number listed in the blame view for forensics purpose, incredibly useful.
The app would be programmed in Swift of course, and at the time I was not yet immersed in Protocol Oriented programming and Dependency Injection so I planned to code the whole thing using traditional OOP programming approach and View Controller containment as Apple used to advocate.
In parallel I would have to program a backend as a destination to all those social networking calls. Being a self taught iOS programmer I have no experience with any of the more advanced server side stacks. I do however know how to make a functional backend using PHP and MySQL, so I planned to service the incoming client requests (routing, query, render) using an old fashioned php script hosted at a remote domain with the corresponding MySQL database.
My technique with server side programming using php involves deploying a version of the server on my laptop and pointing to it from the iOS simulator, that way I can debug the server side code using breakpoints in phpStorm.
Because of the subtleties of distributing changes both on the server and the client side project, I defined two different remote instances of the server, one for the customer to which would point the releases I would ship in TestFlight, and one for developers to which would point the development codebase. This would help me practice migration when the project did go into production and I carefully outlined a series of steps required to safely migrate from one version to the next, and to prevent crosspollination between incompatible versions.
The liberated UI
One thing I was eager to demonstrate was making an app whose UI updates were entirely driven by FetchedResultsControllers. I wanted every input, be it server driven or user interaction driven, to be dispatched only towards the local database, and for all the corresponding UI updates to trickle through asynchronously by way of dedicated observers predicated with the corresponding query. That way, I would have no hard dependencies between input and UI. My UI would refresh in real time to the whims of database activity with blissful disregard for specific contingencies, and would be guaranteed to never be stale, while remaining lightweight and interchangeable.
To achieve this result I would need a full scale multi-threaded core data transaction pipeline, whereas I could dispatch all my transactions off the main thread and reap the corresponding UI updates from FRC callbacks on the main thread. With such a design, the app would remain responsive no matter the complexity of the queries. I wanted the app to be silky smooth in its interactions, as to never interrupt the user experience with an unsightly hiccup or jitter.
The culmination of that philosophy is what I call “the liberated UI”. This concept considers that a user should never be prevented from interacting with the UI no matter what the app is doing, and that the UI should remain responsive at all times. Its reversal would look something like having a user do a pull to refresh and block the screen with a modal and a big ugly spinner until the data is loaded from the server. “The liberated UI” seeks to accomplish the exact opposite of that, and to let the user enjoy unrestricted navigation and interaction while the app does the heavy lifting without any interruption. This kind of implementation, as long as it is working of course, is in my eyes the holy grail of user experience. If I can roam freely across screens and features without ever being interrupted, without ever having to wait or dismiss some notification, while getting all the content I love delivered on time and without fanfare, then I truly can loose myself in the app and spend hours in it. A great user experience is like great service, it’s so good that it’s barely noticeable.
Delivering things in batches
The app showcases a bunch of features:
- An onboarding flow
- A user profile
- The “piece de resistance”, the ability to share text or image to a social media feed, and to comment on or like them.
- Several discoverability vectors to navigate the users graph:
- A map which displays users according to their coordinates.
- A global search which fetches results based on name or interests.
- The ability to follow users and navigate lists of followers and followees.
When making a social network you are faced with a sizeable problem:
You have to find a way to download content from the server onto the client, and make each client a full fledged proxy of the server, leveraging its local persistence to display a “slice” of the content hosted on the server.
Thanks to Core Data the problem of accumulating massive volumes of data is not a runtime concern. Core Data and fetched results controllers are all you need to optimize the memory footprint no matter how much data you have synced from the server. You do need to eventually trim your database if the stored quantities become too massive but that is not something I focused on.
However you do need to optimize deliveries of content, because if there is really no limits to how much data you can store on the user’s device (except, you know, the actual size of the user’s hard drive), there is however a limit to how much data you can parse at a point in time (because all the parsing of server payloads will result in memory consumption). I like to use the analogy of unloading a trailer into a warehouse. The trailer is the remote DB, the warehouse is the phone’s hard drive, and the forklift is the phone’s memory, it can only unload a couple of barrels at a time, but there’s really no imminent limits as to how much it can transfer from the trailer to the warehouse, given enough time of course.
Having already worked on an outlook client before, I was familiar with the concept of sync keys. I used sync keys throughout to batch content in both directions, going forward in time, and going backwards in time. Along the same lines, search results include a paging key for the client to store and to produce on any subsequent requests. As such I was able to download contents in reasonable quantities and was capable of indexing into lists in both directions, by making as many additional trips as required, using my sync keys as proof of work between the server and the client, according to the user’s actions.
When you make a social network you also need a way to refresh threads such as conversations going on in the comments, or lists of followers/followees. I thought at first of leveraging the lifecycle methods of my Core Data entities, hoping that I could have them synced as they went in and out of memory, but that proved fraught with peril, and I settled for a staleness index, at the expiration of which I would sync the item with the server. The ultimate goal was to have that staleness factor be proportional to the popularity of the underlying thread, to throttle the round trips according to the frequency at which content was being generated, but I didn’t get to that.
470 hours of work and 10000 lines of code later, I was feverishly putting together the finishing touches on the last remaining feature of the MVP, the ability to follow, unfollow, and navigate the corresponding lists and sync the corresponding counts. I had promised I would deliver the final version before the big Trade Show where the client had reserved a booth to introduce Dance Love App to the world. There were a whole batch of additional features in the pipeline though, the monetization features, and this adventure could have went on for a while. But unfortunately a growing frustration with the partnership was starting to take its toll, and I just was not willing to continue sacrificing my free time, skills and talent under the same conditions.
Nonetheless, as far as validating my assumptions with regards to process and architecture, I count the experience to be a resounding success. The “asynchronous UI updates” model delivered on all its promises, with uninterrupted interactions and non-intrusive data updates. There certainly was some complexity involved in untangling the wires, especially for the more complex screens which involved multiple types of content across sections, such as the main feed, but overall the experiment proved that it’s surprisingly easy and reliable to bind a table to a query and be done with it, as long as you have the multi-threaded database pipeline to conceal the churn of the database transactions.
The scale of the work would have not been possible without the following things:
- The unit tests, mostly anchored towards verifying the integrity of the server logic, especially when it came to refactoring large chunks of logic which happened quite often.
- Deliberate and detailed documentation of the important aspects of the architecture, including all http contracts but also the steps involved in migrating from one version to the next (database schema versioning also).
- The project management and traceability achieved with JIRA, this allowed me to coordinate with the client and give him a window into the current progress, while being able to delegate work to my development buddies, and produce a tamper-proof ledger of the history of the project.
Will Dance Love App ever succeed, will the UXers ever fulfill their dream of transforming the dance fitness community? I certainly hope so. Regardless of the outcome, I am grateful for the opportunity to have been part of the adventure and derive a sense of accomplishment from having delivered on my commitment without ever jeopardizing quality.
You can find the app here. Don’t hesitate to hit me up with comments and questions.