Project Directory

In a recent lecture, we were taught about the importance of properly preparing project files for handing them over to a client. This has been mentioned previously but here I will go into more detail about the directory itself.

Below is a screenshot of Terminal. I navigated to the directory and typed ‘ls’ which lists all of the files and sub-folders inside. Here you can see the initial organisation of the files, and the essential readme.md which contains more details about each of the files and folders. Below that I used a wonderful homebrew function called ‘tree’ which creates a file tree of the directory and all of the subdirectories and files. As there are a lot of files here, it only shows a small snippet of it, but you get the idea. The indentations are used to show the nesting of files and folders so you can easily track their locations and parent directories.

Screen Shot 2015-05-17 at 11.03.49We have tried our best to minimise duplicate files. This is essential so that there is only one location of the file as to not cause any confusion when working collaboratively. However there are some unavoidable duplicates in this directory. When we copy images into the application, it then makes a copy of the file in its directory, as well as the original inside the assets folder. This is pretty much unavoidable as all the files the app needs need to be in its directory. It is important to remember that if someone decides to update one of the image assets in the assets folder, they then need to update the copy of the image in the app folder to keep it up to date.

Programming: Source Control

Throughout this whole project, we made sure that we used source control (otherwise known as version control)  to keep track of all the changes and files used in the app code. Recording these changes allows you to jump backwards to older code, incase something breaks or the project files are lost. It allows you to compare changes over time and track the progress of the project as it evolves.

Using source control in todays day an age is an essential professional practice as it allows you to easily work collaboratively with other people – as was done in this project with two programmers.

We put our code in a private repository on Bitbucket. This repository is on a remote server and can only be accessed by people who have permission. This is great when you want to keep your code/ project a secret, and is something we had to do as mentioned in the contract signed at the beginning of this project. We use this repository as a backup of the code. When either one of the programmers makes any changes, we are able to push them up to the repository where they will be saved.

Below is a screenshot of our Commits page on the bit bucket repository. Here you can see all of the individual commits made and pushed to the server. A commit is made after some lines of code are changes, a comment is then added so we know what the changes were in the commit – making it easier to find where to jump back to if we ever need it. It also helps us keep track of what has been done so we don’t waste time repeating processes. If you were to click on one of the commits, it would take you to a new page were you can easily see which files were changes and which likes of code were edited.

Screen-Shot-2015-05-16-at-12.57.30

Locally, this source control was used in a couple of ways – either through Xcode itself or using Terminal. Inside of Xcode it has options for you to commit changes, push changes to the remote repository or pull changes from the repository. This is all done with a pretty graphic interface making it nice any easy to use. My personal favourite way of doing it is through Terminal, using the command line to do the same functions to source control my work.

Below is a snippet of the same commit log above, but presented through terminal.

Screen-Shot-2015-05-16-at-12.56.12

To be blunt, using source control for our code helps us meet the learning objective:

02 Originality, creativity and professionalism in the interpretation of a live brief, production conception, pitching, management and realisation of interactive media artefacts;

It is a professional practice which allows us to work swiftly (pun definitely intended) and collaboratively on this project. While I believe we did a fairly good job at making regular commits for the changes to the code, they could’ve definitely been more regular if it was possible. One of the biggest downfalls is that a lot of the code was only written on Thursdays and Fridays while we had workshops. Within these workshops we were able to quickly get  help as and when we needed it, rather than searching endlessly on the internet wasting valuable learning time. As a lot of progress was often made in short stretches of time, remember to stop and commit the changes wasn’t done as often as it should, and would usually be done at the end of the day after too many changes were made to accurately keep track of. That being said, source control has proved itself invaluable to this project and was an essential part of the whole process.

Programming: Bug fixes

Today we encountered a large problem when trying to load the app onto an iPad for testing. The problem was that the app was using too much memory for the iPad to handle loading the app. We were testing on an iPad 2 which only has ~1gb of RAM. We had to significantly reduce how much memory it was using to get it well below this 1gb threshold as the iPad still needs to run iOS as well as the app.

Once the app started up, it was using 1.53gb of RAM to run it. We never noticed this problem before as it never had an issue running on my laptop during the development as it has 16gb of RAM. During our short time learning how to program for iOS we weren’t taught about this kind of issue and what to look out for.

Xcode has some built in debugging tools which can be used to find issues within the app. With the help of the swift magician, Marc, we were able to find where about the problem was. We saw that as soon as the app opens, the memory spikes up to an exceedingly high value and they stays there for the duration of the app running. This make us think it was something to do with the Magna Carta image in the scroll view. Our suspicion was confirmed when we changed the image file to a far smaller one and it was using significantly less memory.

As mentioned in a previous post, we already had to compress the Magna Carta image as the original file was far too large. This file was compressed down to ~188mb, about half the size of the original. Apparently this image is still far too big. When loaded into to the scrollView to display the image, the memory needed vastly increases to approximately 3 times the size. We’re not entirely sure why it uses so much memory, but if we reduce the image file size enough, it will be able to run on the iPad.

The original image we were using was using over 60,000 colours. This gave us a good level of fidelity when zooming into the image. Unfortunately we had to vastly reduce this amount to compress the image to make it usable in the app. We reduced the image down to 128 colours, saving 73% on file size. This reduced image size meant that the app only used ~380mb of RAM to run. While this is still very high, it is just small enough for the iPad to be able to run it without crashing.

The client has said that they will be purchasing an iPad so they they can use our app in the exhibit. We are hoping that they will be buying one of the newer iPads as they have more memory than the one we’ve been testing with – meaning that the app will be more stable while running.

Programming: Final Changes

As we come up to the final week, we’re just adding the final touches to the application to make sure that it is the best product we can provide for the client.

Last week we were given the final translations and contextualisations to use in the app from the client, so the first step was to add these into the code. After we did this, we noticed that they were significantly smaller than before. Because of this, the detailView seemed far too tall compared to the text inside it.

Originally the detailView was 500 high. This value was just an estimate size as it stuck throughout most of the development. First we tried lowering this down to 300. This worked well with some of the shorter clauses as the box didn’t drown them, but with the preface and suffixes, it seemed to be far too small and required too much scrolling. We went for the final option of 400 high. This was a good middle ground between being small enough for the shorter clause translations and big enough for the larger ones.

While doing this, I went through and cleaned up the code. This meant adding comments to functions and deleting excess line breaks to make it neater and easier to read. Commenting the functions and code is essential when handing the project over to someone else, and is always good practice to do just for myself. It is a way of always knowing what the code does and why it works/ was included. Sometimes what seemed like a simple thing to write at the time can seem complex and difficult to understand a week later. Commenting the code speeds up the workflow and is a good professional practice to get into so the code is clear and easy to read, edit and understand.

I also removed the swiftyJSON.swift file from the code. This remained in there from when I experimented with using JSON to get the translations into the code. There were also some old image files left in the project folder from when I was testing for the best solution and images to use. It is good practice remove excess files and text to keep file sizes to a minimum. Its not good to bulk out the project with assets and code which aren’t even being implemented as it makes it cluttered takes up unnecessary space.

Programming: Animations

One of the big things in iOS is having small animations for transitions. These seemingly insignificant effects make a great different to user experience within the app.

First up was the detailView. This is essentially a view which we translate its y coordinate from off screen to on. The original for this was animating with duration. This a made the view smoothly slide up from the bottom. We changed this to be a spring animation so that when it popped up, it bounced at the top of its appearance. This made it a little more fun to use and looked a lot better in our opinion.

I mentioned in another post about making the overlays fade in and out. When a category filter is chosen from the menu, the ones not in that category fade out and the ones that are in the category fade in. This is done by animating the alpha channel of the overlays when the hide or show functions are run. Similarly, when a clause is chosen to show more detail, the ones that weren’t selected fade out, and fade back in when the detailView is closed.

All these effects can be seen in the gfycat video below.

http://www.gfycat.com/YellowishFilthyEyra

UPDATE:

Upon playing with the app, I noticed that if you were zoomed in and then were to change the category, you might not see that anything on the app had changed as the overlays update of screen. I decided to add an animation which zoomed out the view when a category filter is chosen, this forces the user to see the changes and gives a smooth transition back to an overview of the document.

Here you can view it in action, apologies for the sideways video and shoddy filming (it was done on my phone to make things quicker)

http://www.gfycat.com/DearUnitedHylaeosaurus

Programming: Filtering The Clauses

As Magna Carta is a solid block of text, one of the ideas we’ve had from near the start is to be able to filter which overlays are on the screen. The main aim of this was to increase usability and give the user an opportunity to read certain clauses that they might be interested in (based on which filter they choose).

Below is an image of the initial state of the app before starting this process. It shows how it looks with all the clauses highlighted. While we think it actually looks pretty cool, it can sometimes be difficult to touch the icon you want without zooming in, so adding the filter option should help with this problem.

Current state of the app
Current state of the app

To filter the overlays, we wanted a menu that the user could pull out from the side, and it would be placed on top of the Magna Carta image. To do this we used a sideViewController (loosely following a tutorial here). The sideViewController is a separate viewController, but when it shows, it only shows on part of the screen on top of the current viewController – funnily enough this is exactly what we wanted.

To start with we started with playing with icons to act as a button for showing the menu. While doing it, we thought it could be interesting to use the cross icon from the assets made by RedBalloon – keeping with a consistent style. We put it in and realised it didn’t look as good as we had hoped. We also thought that users might not realise it is actually a button used to show the menu.

Testing the Magna Carta cross icon
Testing the Magna Carta cross icon

After that we went back to our original idea of using the more traditional burger icon and it was just as beautiful as we had hoped. We felt the burger icon was the best choice to use in the app as recently it has become synonymous with representing a menu or to show more details, which is what we wanted it to do.

Testing the Magna Carta cross icon.
Testing the Magna Carta cross icon.

When the sideViewController is first set up, it consists of a blank tableView. We added some colours to our tableView before we screenshot the blank version (We didn’t think it would be worth it to go back and remove the colours, so just imagine what it would look like in white).

Empty splitViewController
Empty splitViewController

Now it comes to the fun part – filling the tableView with information!

The prototype cell (and the colours) was set up in the ever difficult to use storyboard. While we had done many of the other things programmatically (The detailView for example), neither we or our tutors had worked with the splitViewController before so it would be easier to let Xcode hold our hand through the process. We added a label which would be used to hold the title for each filter. This label was given a unique identifier so that it could referenced in the code to populate the table.

Screen-Shot-2015-05-11-at-11.21.48

An array of strings was created with the 15 categories used to filter the overlays, [“Key Clauses”, “Church”, “Feudal”, “Forest”, “Jews”, “Justice”, “King’s Officers”, “Miscellaneous”, “Money”, “Peace”, “Trade”, “Wales & Scotland”, “Women”, “all”, “Hide Overlays”]. Similarly, another array filled with the numbers 1-15 was created. This array would be used to references the images. All of the icon images files were names 1.png, 2.png etc. I did this because of a certain way that Swift works. When accessing an image file, the extension doesn’t needed to be added to the end (i.e. 1 would be associated with 1.png) – a specific advantage over using a .jpg file in the app. The .png files also allow for transparent backgrounds which is what we needed.

A function is used to make the length of the table the same length of the array. This make it easy if we ever need to add of remove a category from the array – making it more responsive to change and is good practice to get into rather than setting static values. After this, another function iterates through the array/table and changes the label text to be the same as the current index of the array. A similar thing is done to place the images next to the text label.

After this a function is used to similarly iterate through the table and detect which one is selected when touched. Here is where a specific function was added to get the filter action to work.

Getting the icons and text into the table.
Getting the icons and text into the table.

Turns out, it’s more difficult than we thought to get one viewController to communicate with another. This is where we had to show creativity and originality in our programming. We wanted to use the sideViewController to do something it wasn’t supposed to do. The usual implementation would use the menu to travel to different view controllers. However, we wanted a function on one viewController to affect something on another.

To do this we had to use NSNotificationCenter (following this tutorial). This feature works similarly to a radio broadcast tower. One view is set up to transmit a unique signal and can send data, then the other view is set up to recognise and receive this unique signal then do something with the data it receives.

The first step was creating a special notification key – something unique to this app and the data it was sending. For this I used:

let mySpecialNotificationKey = “This category was chosen”

I then made a function which would be used to send the data across. This function takes a specific type of data, an array of strings. This string is the data used to filter the clauses. I’ll get into more detail about why its an array later.
Screen Shot 2015-05-11 at 11.41.48

In the same function that detects which row is selected, the function is called. When a filter row is selected, the string from that row is send off in the function as the piece of data. For example, if the user tapped the “church” category. the string “church” would be sent off by the function to be received by the other viewController.Screen Shot 2015-05-11 at 11.42.01

This line was then added to the original view so that it can listen out for the special notification key. Once it hears the key (when a filter is chosen) it will run a specific function as a reaction.Screen Shot 2015-05-11 at 11.48.20

This is the function it runs. It has to convert the data it receives back into an array of strings so that it can be used. It then places this information into the onlyShowClausesWithCategory function so that it can search for which clauses have this category, then show them. as it is an array, it needs to know which part of the array to select, so .first is added to the end so it always selects the first item in the array. The array will always only have one item in it because only one category can be selected at once. I added a print line in the log so I could see when/if the information was being sent across.

Screen Shot 2015-05-11 at 11.48.26

Here is the onlyShowClausesWithCategory function which is used to filter the  clauses. It looks very complicated – it is – but I will try my best to explain the basic premise of how it works and why it is needed. The function accepts one value, which is the category. This is the category which will be shown when selected. Inside the function, two arrays are made, overlaysToHide and overlaysToShow. As you can probably guess, this function will sort though the overlays to decide which ones are in the right category to be shown and which aren’t. The for loop iterates through the clauses and overlays, and searches the clause “category” array for the right category. If it finds the category in the array it will add it to the overlaysToShow array, and if it doesn’t, it goes into the overlaysToHide array. The arrays are then iterated through an animated, changing the opacity with a duration so that the hidden overlays will fade out, and the show overlays will fade in (if not already shown)

Screen Shot 2015-05-13 at 12.41.39

Similar to this, a function called onlyShowClause was made which has a similar sorting feature. However the difference here is that it is run when a clause is selected to see more detail. What this does is makes the ones that aren’t selected hide, and while the selected one remains. Below the two images show the Key Clauses category. Then the preface (first block of text) was selected which caused the other clauses to hide so that only one is shown.

Screen-Shot-2015-05-13-at-12.49.39 Screen-Shot-2015-05-13-at-12.49.47

We really liked this feature as it made it even clearer which clause the user is looking at. However there is one minor issue with this. If a clause is selected in the lower half of the screen, it be seen as still selected as the detailView is on top it of. At this point we don’t have enough time to come up with a work around as there are many other things that need fixing/ making and are more important than this small cosmetic issue.

If there was more time we would be able to play around with different ways to mitigate this issue. For example, we could make it so that if a clause on the lower half of the screen was selected, the detail view come down from the top instead. This seems simple enough but would require a lot more thinking and programming to get this to work.

Programming: Changing how the overlays work

Up until now we only ever had one overlay for each clause. This was because we didn’t actually know where the clauses were on the document so we built it using this method. After we got send the information of where the clauses were, we were able to mark down the coordinates and size of each overlay. As some clauses span over multiple lines, it meant that multiple lines per clause needed to be drawn – completely changing how the app functions.

Location of all the clauses on the document
Location of all the clauses on the document

Each overlay has an x & y coordinate, and a width and height. Each clause may consist of up to 4 overlays. Each overlay in each clause must go to the same detailView for the correct clause details. These were the basic steps of what was needed to get this to work.

First step was to create a “Clause” class,  this has a name, a category, a colour, and the details in the detailView. Similarly, an “Overlay” class was needed which would work with the Clause class. The Overlay class has a member of Clause, where it gets it’s colour and so it knows which clause it is associated with. The overlay is also given a frame, the x & y, width and height, so that the box can be drawn on the document image.

In the main viewController an array is made where the clauses are mapped to the overlays. Before, when there was only one overlay per clause, the array was iterated through adding a tapGestureRecogniser to each overlay so that each go to different clauses. Similarly here, the overlays are iterated through and added to the clauses (I’m not entirely sure how this works, but I have a vague understanding. Marc helped a lot with this part as it was very confusing, and needed to be done quickly due to the deadline).

Screen Shot 2015-05-13 at 12.22.01After this is was a matter of filling the arrays so the overlays and clauses can be made. Here is an example of what it looks like:

Screen Shot 2015-05-13 at 12.30.40At the top, two variables were made to store the colours. This was done for two reasons: Firstly, to make it easier so the RGB values didn’t have to be repeatedly typed out, and secondly, so that the colour values could easily be changed without having to do it for each clause. As you can see by the comment, at this stage the colours aren’t fully decided on; we have something close to what we want but as they have different opacities, it doesn’t look precisely what we want it to look like. Minute attention to detail like this is a good professional practice to have so that small mistakes aren’t repeatedly made.

Each clause in the array has a number of details. The name is used for the title in the detailView, the category (often categories) will be used to filter the clauses once this part is created, the detail is the body text for the detailView and the colour is the colour of the overlays for that clause. Below that comes the overlays for which CGRects are used – similar to what we had before but this new method allows us to have more than one.

Programming: Working With JSON

Today, I explored the possibility of retrieving the data from a server. This was experimented briefly before, but as the final idea developed, it seemed less relevant.

However, I had the idea that rather than filling the app’s code with all of the text for the details page, it could be fetched from a server instead. To do this I used a Swift extension called SwiftyJSON. This significantly simplifies the process of fetching data from the server, making the code work better and much easier to read, write and understand.

To start with, I created an array of strings to store all the detail text fetched from the server. The array is made outside of the function used to fetch the data so that it can be used globally, rather than only inside the function. The function, loadDetailText(), is what we call to do the JSON request. The function gets the .json file from a server (in this case my web server as a test) and then reads the data it collects. I used a loop to iterate through the JSON array, and used each iteration to fill a space in the local detailArray.

Screen Shot 2015-05-03 at 18.15.17

Below is an image of how the JSON file looks. It is set up as an array of detailText with each detailText having a string. These strings populate the detailArray allowing us to access them individually when needed to place the text in the app.
Screen Shot 2015-05-03 at 18.15.42

At the moment, we are unsure which approach would be best for the application, either embedding the text into the app, or fetching it from a server. Embedding into the code currently seems to be the best idea, as it means an internet connection isn’t required to use the application. However this experiment with JSON is still valuable as a fallback if ever needed.

Programming: Using the official Magna Carta image

After weeks of waiting we finally got sent the real image of Magna Carta which were going to use in the app. This image was a much higher resolution than the one that we were using currently, which we got from the Salisbury Cathedral website.

The Cathedral had this picture taken specifically for the 800th anniversary of the signing of Magna Carta so it was essential that we used inside the application as a springboard for the navigation of the causes.

There were some technical limitations with the image that they gave us. The original image was over 34,000 pixels wide which would allow the user to zoom in and see details in the document that could never be seen with the naked eye. Due to the large image size, the size of the file is also incredibly high which would make the size of the application much larger. For us, however, this isn’t that much of a problem at this point as the application will only be used by Salisbury Cathedral and would not be available for general members of the public download.

We tried putting this image straight into the application but due to its large size, the iPad simulator wasn’t able to ever load the image. This meant had to re-evaluate our approach to having a large zoomable image inside the app.

We put the image into photoshop and resized it down to 8000 pixels wide. Then we were able to compress it while saving it as a PNG file as to not lose too much of the image quality. While being significantly smaller than the old image size, it still allowed the user to zoom in considerably, and see more detail than could be seen before. While the client may not be too pleased with this limitation, it is something outside of our control due to our lack of programming expertise and the limitations of our methods.

It was suggested to us that, if we were much more competent with the software, we would be able to load the image as a series of tiles. This would mean that only the tiles that are on the screen would be loaded at any given times, thus reducing the work load on the device and allowing for a much larger image size. This would work similar to how the SWIF mapkit loads each piece of the map when requested. However due to our inexperience, and the time constraints of the project, this is not something we are able to explore.

As we now have the size and resolution of the image we are going to use in the application, we can now get started on finding the locations of each clause within the document. Our client has provided us with an image demarcating the start of each of the 62 causes allowing us to translate this information to our new image. This process will require us to find the X and Y corners of the top left corner of each clause as well as the width and height. These are the details needed in the swift code to allow us to draw the overlays on the app. In some cases a calls will go to meaning that it will require more than one rectangle to be drawn. As it stands we currently don’t know how to solve this problem in the code but it will be something that will be looked at in future prototypes.

Programming: A new approach to the detail view

Previously, we the details for each clause stored in a different view controller. This meant that when one of the clause overlays was tapped, the view would change to the new controller, giving a different full screen page which would have the details about the clause on.

This is what it looked like: https://gfycat.com/LateFalseAracari

After having a meeting with the client, to show them our progress thus far, he had a different idea for how we could display the information. Instead of sliding across from the side, he’d like it so slide up from the bottom, overlaying the Magna Carta image.

Doing this meant we had to take a completely different approach to the way we were doing it. We started by deleting the old view controller, it worked, but wasn’t the right thing for the job. To get the detail view as an overlay, we created a new UIView which would appear on top of the image view (Magna Carta image); this would be the container for the detail view. To start with, we made this view to be 500px tall, so it take up around the bottom third of the screen. This size is likely to be updated as we fill it with information to better suit the shape and size of the text filling it.

We decided to do this programmatically rather than using the interface builder as it would allow us to have better control over positioning elements and it made it easier to keep track of animations and functions. A lot of this was due to personal preference. As we have struggled with using the interface builder in the past, we opted for using the route we were more comfortable taking.

We created an instance of the detail view, giving it a position and a size where it would appear off the bottom of the screen. A boolean was used to keep track of weather or not the detail view was showing. When an overlay is tapped, the boolean switches from false to true, causing the detail view to animate into position on screen.

With this there were a few inherent problems which would need to be solved. The overlay would show and hide when tapped perfectly, when the image wasn’t zoomed in. Once you zoomed into the image, the position of the box was completely wrong and nonfunctional, as shown in the gfycat video below.

http://www.gfycat.com/BeautifulMassiveHypsilophodon

This was a problem due to how the view,  image view, the scroll view and the subviews were set up. Pretty much how everything so far was connected, was wrong. This meant changing a few lines of code to fix the way the different views were connected. The image view needed to be a subview of the scroll view so that it could be zoomed and scrolled. The clause overlays also needed to be in this scroll view so that they would move with the image and stay in the correct positions. We then had to put the detail view as a subview of the view, rather than the scroll view so that it would appear separately and not be moved or zoomed with the image and other overlays.

This solved the problem with detail view overlay allowing it to appear separately from the scroll view and be immovable.


UPDATE: In the first instance there was very basic functionality when interacting with the clause overlays and the detail view. For example the only way to close the detail view was to tap on one of the clause overlays again which would flip the boolean to false and make it hide. There was also a problem that if, for example, clause two was showing and the user then tapped clause 3, the information in the overlay would change to the new clause but it would still close the detail view (again, because of the boolean).

This is why we added a few new functional features to make the interaction a lot more intuitive for the user. By adding a nested if statement that meant if the detail view was already showing, and if another clause overlay was tapped, it would just switch to the right information for that clause rather than close the view. At the moment this switch is very sudden, and we think it might not be very clear to the user that the information has updated. In the future we plan to add animations to the text so that it updating it a lot more visible and aesthetically pleasing.

We also added a swipe gesture recogniser to the detail view. As mentioned before, using built in gestures makes the app more intuitive to use. We wanted to use this feature by having a swipe down gesture to hide the detail view; functionality that can be seen in many other apps.

We also added a tap gesture recogniser, to the image view. This meant that if the detail view was open and the image view was tapped it would again, dismiss the detail view. This again is another functionality feature which can be seen in other apps. When there is a view present on top of another, tapping the one behind often dismisses the top one so the behind one can be better seen.

Adding these little gestures has made the app instantly feel much more like an iOS application. It is the addition of small, seemingly insignificant features which make a large impact on user experience, making it far more pleasurable and enjoyable for all.

Below is yet another gfycat video of the app so far. The overlays are different to what you’ve seen before as there are a few incomplete changes which need to be finished and will be blogged about in the future. The video shows the new interactions, albeit very quickly and rather difficult to follow. It shows opening a clause’s details (And then accidentally closing it), then opening a clause, switching to a new clause then swiping down to close the details. Then finally opening a clause and tapping the image view to dismiss the detail view.

http://www.gfycat.com/WiltedDefensiveHarvestmouse