iTunes U

My only real problem with this method has been lack of access to the lecture itself. iTunes U has been around for a while, with many schools offering access to videos and notes classroom lectures. Its been gaining steady traction, to the point where I’m pretty sure you could do an entire undergraduate degree online.

There’s even an app for iOS that allows you to access all the content on your mobile device alongside your computer. Speirs has a great article discussing iTunes U, if you’re really interested in all the technical details.

Currently I’ve got one monitor up playing the lecture, with slides up on my iPhone (here’s one thing I’d love an iPad for), and a browser and programming tools up on the main monitor. Granted, this whole solution isn’t as beneficial as personal access to a professor to answer your questions and make sure you’re learning the material. Then again, its a heck of a lot cheaper.

On the whole, the experience has been very reminiscent of sitting in a lecture with my laptop open. Including the unfortunate point where I get distracted for a few moments and suddenly snap my attention back to the professor, trying to remember what he just said. Unlike an actual lecture, I can quickly rewind so I don’t miss anything.

Video Scrubbing

I’ve been watching a lot of videos with Plex lately, and I’ve gotten hooked on its fast forward and rewind keyboard shortcuts. Namely, the right arrow skips forward 30 seconds, the left arrow rewinds 15 seconds, the up arrow skips forward 10 minutes, and the down arrow rewinds 10 minutes. Much to my dismay, I hit a giant brick wall when trying to do something similar in iTunes. It just doesn’t have any built in functionality for this. Scrubbing using the progress bar and mouse is doable, but nowhere near as fast, and requires careful attention.

Awkward half solution

If I view the content on my iOS device, there is a dedicated rewind 30s button. This works great, but watching hours of lectures on my iPhone is a bit much. I tried using AirPlay to stream it to the TV, and it worked out ok. That said, I don’t really want to be in front of the TV all the time, so this is out. In iTunes, if you change the Media Kind for each of the video files from iTunes U to Podcast, suddenly the 30s skip backwards button appears on the computer. So I’ve got skip on the computer, but in a severely limiting form. I can do better.

Creating a Solution

My final solution to the problem was of course, programmatic. I started out by opening Automator and creating 4 new Services: Forward 30s, Rewind 15s, Forward 10m, Rewind 5m. Each Service will only work with iTunes, and takes no input. The script for each service differs only in the first line. The script simply changes the player position of the current track based on the skipAmount value.

Change the skipAmount property to suit your needs.

Then, hit up System Preferences > Keyboard > Keyboard Shortcuts and bind the arrow keys to these services.

Bind the arrow keys to the new Services

While my initial script worked fine, I made a few refinements. First, I check for a current track, which gets rid of an error message if you press a hotkey when there is no track playing. Second, I added checks for skipping before the start or past the end of the track, which was causing the player to exit when this happened.

Final Product

The ¬ means that a long line is being broken up into multiple lines.

And here’s the same code for cut and paste:

property skipAmount : -300
on run {input, parameters}
tell application "iTunes"
if current track exists then
if (player position + skipAmount) is greater than (start of current track) ¬
and player position is less than ((finish of current track) - skipAmount) then
set player position to (player position + skipAmount)
end if
end if
end tell
return input
end run

Now I’m able to scrub through lectures with ease. One can always hope that Apple will bake in this kind of functionality, but I doubt it. Until then, I’ll make due with clever hacks.

It all started with this

Micromanging your Phone

When queried, the gentleman said the phone hadn’t gotten wet, and it was fully charged, and he was not aware of how to reboot the device. The sales associate plugged the phone into a dock cable (apparently it was very low on power), and the phone responded just fine. He then proceeded to, without any further investigation, tell the gentleman the problem. “You’ve got too many apps running. You have to double tap the home button to bring up the task manager, then tap and hold on an app to make them wiggle, then tap to kill them. You’ve got to kill your apps when you’re not using them, or the system will get full and freeze up.”

I was very shocked to hear this, because I actually know how iOS manages its memory and processing power. Clearly this sales associate didn’t have a clue. I tweeted and walked away amused, only to have a very long twitter conversation with a friend about this issue. Apparently he had been told the same thing, but by a genius. To make matters worse, he had shared the knowledge of how to kill tasks (which by design is pretty difficult to stumble upon) with a friend of his. This friend now runs an app, and when he’s done, immediately kills it. Every single app, every single time. I was just flabbergasted.

I mean, I kill apps myself, but only in one specific instance: if the app has frozen and its quicker to kill it and restart than wait for iOS to decide its had enough and perform the kill itself. The thing is, I know what I’m doing. I would never tell a normal user how to kill their apps, unless I also included a hefty educational speech about when and why you do this. In everyday usage, nobody should have to do it, period. Even those rare instances should be infrequent enough that the average user doesn’t encounter it. Having a user forced to manage what apps are running strikes me as straying far into Android territory. If iOS is failing so badly at memory management that the everyday user is forced to manage running processes, why not add in an activity monitor app so I can see memory and CPU usage while I’m at it.

The Problem is Getting Attention

I spent some time poking around the dark corners of the internet, and there were plenty of confused normals spouting this “kill your apps manually” nonsense. To make matters worse, the people who knew better came off as condescending when trying to correct the normals, and every thread I found just devolved into name calling. I didn’t feel up to wading into the mess, so I made sure to educate my relatives, then let sleeping dogs lie. Fast forward a few months, and the problem is getting a little attention, at least in certain circles.

It started with Speirs, got picked up by Gruber. Then Speirs went and documented the whole process on video, which in my mind should lay to rest this issue, at least for anybody willing to watch the video in its entirety.

Who is To Blame?

The thing is that really bothers me is, how exactly are “normals” getting educated about the app kill functionality? I’m not sure, but my best guess is word of mouth many times removed, eventually traced back to uninformed apple employees or know it all children.

Clearly Apple needs to include the app kill method as part of their employee training, and make sure their employees know what it does and does not accomplish. That would at least stop the misinformation at the source.

I know Apple would be happy if most users never knew about the app kill functionality, and perhaps its best that way. However, in order to clean up the mess made by their well intentioned employees, consumers need to be educated on the facts, at least on a case by case basis. People who overuse the app kill functionality are wasting massive amounts of their time, and this needs to be stopped.

This isn’t necessarily a platform specific set of problems, as I’ve seen each issue in both Windows and OSX. When a game is multiplatform, however, it’s almost a certainty that the Windows version handles multiple monitors better than the OSX version.

Honestly I’ve only seen one company, ever, get multiple monitor support right. And as a bonus, they do it right on both Windows and OSX, and they have a full screen windowed mode in one of their games that is just absolutely perfect. I’ll give you one guess which company that is.

Civilization V

I decided to kill a little time the other day, and I didn’t feel like booting to Windows to play Civ5 in all its glory. The OSX version usually suffices, and performance is ok as long as I keep the map size reasonable. While attempting to play, I quickly remembered why I hate playing Civ5 on OSX: resolution issues.

I have multiple monitors, and I want the second monitor usable at all times. That’s kind of the point of having multiple monitors. If I play Civ5 in fullscreen mode, it will blank my secondary monitor, so this is out. Fullscreen mode games also tend to be very slow to react to switching between the game and other applications, so I usually play in windowed mode. Native monitor resolution windowed mode has a slight performance hit over fullscreen mode, but its acceptable.

Problem

Before

This wouldn’t necessarily be a problem in many games, but Civ5 has a TopPanel which is an in-game menu bar, offering access to critical gameplay information. Much of this information you just cannot access anywhere else.

No toggles for you!

So basically my options are:

• Reboot to Windows and play fullscreen with proper multiple monitor support.
• Play fullscreen in OSX with one monitor blanked out.
• Play fullscreen windowed in OSX without access to critical game information.
• Play windowed in OSX at the second highest resolution (1920×1200).

The latter option seemed to be my only real choice, and it would give me a small performance boost as a side benefit. I do dislike the loss of game immersion by playing in less than fullscreen. It also irks me to waste all that monitor space.

Solution

After

Since the UI is moddable through Lua, I figured it might be possible to write my own mod to change the position of the TopPanel. I poked around a bit trying to do this, to no avail. I managed to add some custom content to the TopPanel (by editing the ~/Library/Application Support/Steam/SteamApps/common/sid meier's civilization v/assets/UI/InGame/TopPanel.lua file), but I couldn’t figure out how to change the bar’s position. Luckily, while fooling around with this, I stumbled upon a real solution.

In the file ~/Library/Application Support/Steam/SteamApps/common/sid meier's civilization v/Assets/UI/Options/OptionsMenu.lua there’s a little variable at the top of the file called m_WindowResList. Apparently it stores some default resolutions for the game. I found some code later in the file adds your current monitor resolution to the collection (mine wasn’t there by default) “just in case they’re running something weird”, as the developer’s comments informed me. I simply added a new resolution to this list, and removed 40 pixels from the Y resolution.

Select the new resolution

 
Once this is done, you can simply launch the game and select the new resolution from the Options screen. While this solution isn’t perfect, I declare it close enough.

The app was pulling large amounts of data from an Oracle database using stored procedures, and building presentation html right there in the servlet layer. We couldn’t afford software to properly analyze the app, so we wrote a quick timer and started doing simple start/stop events throughout the loading of particularly slow pages, tracking the time it took to perform specific actions.

We slowly figured out that no one part of the app was being egregiously slow; The whole thing was just uniformly slow. Obviously loops through resultsets were taking up the bulk of the time, but inside the loops each statement performed at the same equal snail’s pace.

Searching for help on the web didn’t help. I’m not sure if my google-fu was just in its infancy back then, or if there really was no information out about this problem. Eventually I started writing simple test servlets and tried to do the same task in different ways, hoping to flush out the piece of code that was causing the slowdown. As it turned out, I stumbled upon the solution in Sun’s Javadocs.

The Problem

Apparently in Java, Strings are immutable (they cannot be altered after they are created). The idea is like so: Lets say you’re going to display an invoice to a customer. You have their name in a string, pulled from the customer table. You read in the invoice, and you have their name again, pulled twice from the billing and shipping information. Now you have three string variables all storing the same static bit of information in memory. Rather than waste 3x the space, Java simply points all three to the same chunk of memory.

Now you might think this would be a bad thing; What if the customer changes their billing address name? Because strings cannot be changed after they are created, every time you think you’re changing a string, you’re actually creating a brand new string and pointing your variable to the new string. In the event that the old string has no more variables pointing to it, it will be garbage collected.

Understanding how this whole mess affected this application has to do with how we were building our html. You see, building a page went something like this:

...
String body = new String();
body += "<table border='1'></table>";
body += "<tr>";
body += "    <th>Column 1</th>";
body += "    <th>Column 2</th>";
body += "</tr>";

while( rs.next() )
{
    body += "<tr>";
    body += "    <td>"+rs.getString(1)+"</td>";
    body += "    <td>"+rs.getFloat(2)+"</td>";
    body += "</tr>";
}
body += "</table>";
...
return header + body + footer;

That’s right, every line of html that was being created for every page was done by string concatenation. With strings, when body += "html here" is used, it creates a new string containing the concatenated contents of both strings, then points body to the new string. This means that the old string body pointed to, along with the html that was concatenated with it, are now floating around waiting for garbage collection. Each and every line of html created using this method has the side effect of creating 1 or more new string variables. Object creation is expensive in Java, and the garbage collection for these excess strings was nearly as expensive.

The Solution

The solution was actually pretty trivial, but time consuming to implement. Java has a StringBuffer class which is specifically designed for this type of problem. It stores the same information as a String, but it a mutable fashion. This allows us to alter its contents without the hassle of object creation and garbage collection. The fixed code looked like so:

...
StringBuffer body = new StringBuffer();
body.append( "<table border='1'></table>" );
body.append( "<tr>" );
body.append( "    <th>Column 1</th>" );
body.append( "    <th>Column 2</th>" );
body.append( "</tr>" );

while( rs.next() )
{
    body.append( "<tr>" );
    body.append( "    <td>").append( rs.getString(1) ).append( "</td>" );
    body.append( "    <td>").append( rs.getFloat(2)  ).append( "</td>" );
    body.append( "</tr>" );
}
body.append( "</table>" );
...
return (new StringBuffer( header.toString() )
                 .append( body )
                 .append( footer )
       ).toString();

For anybody doing Java web apps now, you just won’t run into this problem as often. JSPs allow for much more readable presentation code, and you rarely run around appending to your html like this. That said, the misuse of strings in Java is in no way limited to this example here. The problem crops up more than you’d imagine, with experienced and inexperienced developers alike.

New Job, Same Problem

At my second big computer job, I was surrounded by some very smart people. They had been using Java for years, and were currently maintaining a huge, high performance web app using JSPs, Servlets, and EJBs. They were handling exclusively backend work, and passing off finished code webmonkeys to design UIs around. I remember sitting in my little cubicle, pouring over code rapidly, trying to get a grasp of how the application worked. I overheard three of the senior developers discussing a problem a few cubicles over. Much to my surprise, they were having performance problems with a new piece of code that sounded suspiciously like it was an immutable string issue.

I listened for a minute or two, then swaggered over to solve their problem. What took days to figure out at my first job was related and then fixed in mere minutes here, and the performance issue disappeared. I’m always reminded of this moment, and how even experienced developers don’t know everything. There’s always something more to learn.

The Rise of the Syncing Services

Eventually I stumbled upon a service called SugarSync, and I fell in love. I was able to specify certain folders, and these folders would automagically stay in sync across multiple computers and multiple OSs, without effort. I even had access to the files through a website.

I set up a client with SugarSync to mirror their documents to several business computers, and used a script to make backup copies of critical files, which are then backed up to the web via SugarSync. Coupled with its file versioning support, it has on more than one occasion allowed me to recover files and undo drastic mistakes that would have otherwise crippled the business.

I ended up getting into an invite-only service called Dropbox not long before it went public, and to date it is my primary solution to the remote access problem. I still maintain SSH server and remote desktop access to my desktop, but I almost never need to bother.

My roommate recently had her notebook stolen, and was rather devastated. She’s starting a new business, and one of the lost files was the most recent copy of her business plan. She asked me to OCR an older copy she had lying around, and I was a little confused; I had set her up with Dropbox several months prior, and I don’t think she fully understood the benefits. Sure enough, all of her files were fine, safely mirrored to her old desktop and the web.

While these services are highly useful, they have yet to hit the mainstream. Both Microsoft and Apple have their own competing versions, and moving forward we’re going to see this kind of sync become a key part of the computing experience. Already you can log into your online photo gallery or email account from anywhere and have access to all your data. Eventually all of your files will be stored in the cloud, and will be automatically accessed by any device you own. You’ll be able to log onto a friend’s computer, and all of your files, applications, and settings will be the same as if you were using your own computer.

I don’t see computers all becoming dumb terminals like the early days of computing, but the effect will be much the same. It will no longer matter if we’re using a tablet, phone, desktop, or laptop. It will no longer matter where we are, or who owns the device we’re using. We’ll no longer be constrained to any particular device or location to accomplish our tasks.

There are programs out there that allow you to customize your systemwide gestures, but I’ve always been kind of a purist when it comes to input solutions. I’d rather learn to use what ‘the masses’ use, so I’m equally at home on someone’s stock install as I am on my highly customized one. Recent self discovery has made me rethink my position a bit, but I digress.

Up or Down

With the release of OSX Lion, millions of people are quickly discovering that scrolling with the trackpad doesn’t work like it used to. In fact, it works opposite. Years of scrolling with the touchpad, in multiple OSs, has taught us that dragging your fingers top to bottom should scroll the page down, and dragging your fingers bottom to top should scroll the page up. Amusingly, on all iOS devices, the opposite is true. With a touchscreen, you grab and drag the page with your finger. Thus, dragging bottom to top scrolls the page down by pushing the content up and off the screen. Apple has decided to force this scrolling paradigm on its OSX users, and I can’t say I disagree with them. Honestly it takes all of a day to get used to the new direction; Your brain is quick to adapt.

The exception, of course, is if you have to use another computer: An older copy of OSX, or a Windows machine where the scroll directions are now ‘backwards’. Your brain can compensate with a little difficulty, but its confusing to go back and forth every day. The simple solution is to swap your Mac to match the Windows default, but this causes problems moving forward. If you spend a lot of time working on different Macs, you’re going to spend a lot of time being annoyed with the scroll direction. You can of course change the Windows scroll direction to match the Mac default, but you’re going to have the same problems if you end up working on a lot of different Windows machines. The best solution (other than waiting for Microsoft to mirror Apple’s change, or waiting for Apple to capitulate and change back), is to just pick what world you spend most of your time in, and set your scroll directions to give you the least amount of change.

My roommate had this very problem, but was unable to change his Windows scroll direction. His mouse driver didn’t support it, and he lacked the registry access necessary to flip the proper bit. I wrote a fix for him in my head a few seconds after he complained about it:

WheelDown::WheelUp
WheelUp::WheelDown

Running this script in AutoHotkey reverses the scroll direction for almost all cases (ignoring a few games that grab low level mouse input).

Backwards or Forwards

In addition to the scroll direction changes, OSX also changed the default forward/backwards direction for webpages in Safari. Before, three fingers right to left would go backwards a page, and three fingers left to right would move you forward a page. Now, two fingers right to left moves you forward a page (by pushing the old page offscreen), and two fingers left to right moves you backwards a page. My big problem here is with Apple’s Finder application. Two finger forward/backwards does not work, and instead scrolls around the content in the window. Honestly, this is exactly what I want and would expect, but it breaks rank with Safari.

Two Fingers or Three

To make matters even more confusing, there is an option to enable the old 3 finger scrolling gesture, which then allows Finder and Safari to both support 3 finger scrolling in the same direction: the old one. Apparently 3 finger scrolling ignores the systemwide scroll reversal option. There’s also a 2 and 3 finger option that allows you to use both gestures, but it ends up being an ungainly mess. You end up scrolling backwards with 3 fingers, forwards with 2.

Consistency

Honestly I’m not sure what they were thinking here. Consistency is one thing I value highly, and this is anything but. For now, I’ve accepted reverse up/down scrolling, but I’ve kept backwards left/right 3 finger scrolling. A mess to be sure, but its the smallest mess I could make.

The default OSX Terminal application is a great little program, with plenty of features. It even comes with a selection of themes to style the look and feel of the app, and years ago I may have just changed to the Pro theme and been done with it. However, I’ve long since developed my own terminal theme, very similar to the IR_Black theme. I have some mild look and feel differences, and I’ve altered the keybindings to more closely mirror what happens in the rest of the OS. In addition to this, I also copy over a .bash_profile with some useful settings (prompt colors, aliases for ls variants), a .vimrc file with some minor tweaks, and a .inputrc file to help with keybindings. In particular, standardized keybindings for home/end and pgup/pgdown are necessary to preserve my sanity at the command line (yay laptop keyboards).

Only after this ritual is complete can I really settle down and get any sort of command line work done. Some day in the distant future, perhaps someone else besides me will launch the terminal on this laptop and say a little prayer of thanks.

An Easier Way

I’m still a bit crazy about missing news, but thankfully RSS has at least made this process manageable. It amuses me to no end that the average user has no idea that this technology exists, despite the fact that it is used so widely. That said, not every page puts out a feed. Sometimes they do publish one, but its riddled with information you don’t want: Occasional advertisements, rants about off-topic issues, and other information that doesn’t interest you. While stumbling my way to a solution for this problem, I discovered Yahoo Pipes.

Imagine if you could take a source of information, anything from a webpage to a CSV file, and filter out what you don’t want. The resulting data can then be merged with another data source, and then republished as a single new data source. With Yahoo Pipes, you can do exactly this.

Tor doesn’t offer feeds for its authors, so I my first pipe took their master feed and filtered by the author I wanted. I created a pipe to filtered out a friend’s blog posts from a collaborative site. I made a pipe to watch the price change feed for appshopper.com, and filter out only applications I was interested in buying. I even made a pipe that takes a calendar and strips out events I don’t want to attend. You can take most any published data as input, transform and combine it at will, and output it in whatever format you desire. It isn’t exactly the most user friendly of tools, but its got enough power to make up for that.

Overlap

There’s a serious amount of overlap in my feeds; I’ll read about news from its original source, from someone reporting on that source, from someone reporting twice removed, and then it’ll hit mainstream media, and I’ll read about it 10x over the next few weeks as it gets repeated. I like knowing about bleeding edge technology news before the mainstream media does. Often I’ll hear about things days or weeks before they hit CNN. Often rather than follow a pundit and read the news he puts out, I’ll go straight to his sources and follow them instead. I have to constantly prune my sources, lest I become overwhelmed. Do I want to read information from sources that are among the first to report it, or would I rathe read a more critical analysis from a site that reports thirdhand a few days later? Often the best choice still involves some level of overlap.

Filtering

I’ve tried to create pipes to handle this overlap, but its not easy. Assuming two sites publish a story and use the same title, the stories are easily matched and one can be discarded. This rarely happens. Even using lots of metadata (author, source link, rough date/time, categories), its a beast of a matching problem and difficult to do consistently. Also, there’s the problem of comments, which offer actual useful information from time to time. If you discard one article, you lose the comments of all the other duplicate articles.

Clearly this issue needs to be handled at a higher level than pipes was intended to solve. The underlying data to solve this problem is there. Most reputable sites link back to their sources. Brute force comparison of articles can easily establish if they’re discussing the same topic. I want an engine that sees all news from all sources, and groups them together by common threads. For critical topics, an actual person could serve to quickly make links between related stories.

As a story develops, additional sources and comments can be added to the aggregate news source. If I read about a product, and it is mentioned a dozen times over the course of the next month, with each source offering no new information, I shouldn’t have to see any of the duplicate articles. If these new sources offer actual new information, or a decent amount of analysis, I’d like the information to be merged with the original story, and the story marked as updated with new information. No matter what site you visit to learn about a news story, you should be able to easily link to the master source of all news on that story.

Someday soon I hope to have my overlap handled automatically and seamlessly. Perhaps then I’ll free up a few more minutes each day to do something besides reading. Then again, perhaps I’ll just broaden my horizons and start reading new topics.

Update 9/2011

A tool far less powerful than pipes, but quite interesting, has cropped up recently: ifft.com. You can use recipes which allow you to have triggered actions happen on pre-specified events. It has only a small overlap with pipes in terms of functional uses, but you can do some interesting things with it. Feeds can be created from every post you star in Twitter. Instagram pictures can auto save to your Dropbox. Checking in on Foursquare can trigger a Facebook status change. A text message can be sent if the weather report says its going to rain. If someone tags you in a picture in Facebook, you can have the picture emailed to your mom. I’ve found ways to use this service to complement pipes, by creating quick and easy to use data sources that pipes can then work with.

There is no worse sin for an OS than to let an application take control of my focus. Nothing is quite like the exquisite torment of being in the middle of giving your program commands, and nothing happens. Perhaps another program popped up in a second window when I wasn’t looking and took focus from me. Sometimes I launch a program, then switch to a web browser and start navigating, only to have my focus is lost to a modal popup. I switch back to the web browser, continue navigating, and the same app opens another modal popup, breaking up my browsing experience yet again.

Seriously, I’m trying to get some work done here. I swapped away from your slow, bloated program, because it takes 20 seconds to fully launch. I don’t want to sit there wasting time while you finish. Just do your load, and don’t steal my focus, is that so hard?

Besides being horribly annoying, its been known to cause serious problems for people. My personal low involved me setting my away message in my messaging program to my banking website password for two days. Currently most of the onus of this problem is on the application developer, but I think that’s just passing the buck. OS developers need to make a stand and put in place protections to prevent this sort of thing.

This time I babysit iTunes through all the steps in sequence, though this time there’s an extra step 4 in there (unsure how that happened).

Ideas

I know you have some UX guys working for you Apple; Perhaps you could throw them at this problem for a while? My thoughts:

• Make iTunes more responsive. I kept tasking out because of the huge lag between clicking OK and actual things happening.
• Keep my apps up to date, automatically, in the background. At least make it an option. And please, delta sync.
• Get rid of the age restricted warnings. Ask me once, per account (or even per app), and that’s it. Never ask me again.
• Remember where I am. If the user happens to sit at the Enter password to continue box for 10 minutes, comes back, and clicks OK… continue with what they asked you to do 10 minutes ago, don’t just sit there.

Its my sincere hope that somewhere deep in the bowels of Apple, there’s a skunkworks team that has spent years rewriting iTunes from the ground up. Its responsive, easy to use, performs simple tasks with little to no effort, and doesn’t require holding down option to delete a song from within a smart playlist. Any day now, it’ll drop. Please.