Category Archives: Development Thoughts

Development Thoughts

Trying out Chrome’s App Store for the Web after falling down the W3C Widget Rabbit Hole

3 Comments

In my last post I discussed offline storage for web apps, and that I wasn’t sure how users were supposed to know that certain apps worked offline. Google Chrome has come up with a neat solution for this that allows users to “install” web apps via their Chrome Web Store. I’d played around with its previous incarnation, when it was the Chrome Extension Gallery, but I hadn’t really been back since it opened up to web apps and changed its name.

Installed App

The Chrome Web Store is interesting because you can submit both packaged apps and hosted apps. A hosted apps is simply a zip file of a metadata file and some icons. The metadata file explains general information like the URL for the app and if it works online or not.

Installing a hosted app is sort of a fancy way of bookmarking it, but the fact Chrome tells you whether it works offline or not is a big plus. This solves the issue of letting users know that an app’s URL will work even when they don’t have an internet connection.

This is exactly the kind of thing I was looking for! To try it out, I decided to create hosted apps of my Text to ASCII Art Generator and JavaScript Snake Game:

Both work offline, and I plan to create hosted apps of some of my other programs too, but for now I’m just testing the waters. FireFox will also soon launch a web app store called The Mozilla Marketplace. It’s setup will be similar to chrome’s, though it’ll be slightly more feature rich, allowing you to run web apps like Desktop programs. This feature is similar to the idea behind W3C Widgets, though W3C Widgets seem to have fallen out of favor.

As an aside, before I ended up at Chrome’s Web Store, I took a detour through the world of W3C Widgets. I hadn’t heard of them before, and unless the tides change, I may not hear much about them again. However, I figured I’d write up what I learned, since they seem to be one of the least popular pieces in the HTML5 puzzle.

W3C Widgets?

While researching offline web storage, I was bothered by the fact that normal users would have no idea that certain URLs still worked when they were offline. With this concern, I ended up emailing the W3C’s fixing-appcache mailing list because I could only find a small bit of discussion on the topic, and that discussion didn’t come to a consensus about what to do. The response I got was pretty interesting, and made me realize there were areas to the HTML5 world that I hadn’t seen:

This is essentially what the Widgets spec is supposed to achieve.  See
here for an example of a widget configuration:

http://www.w3.org/TR/widgets/#example-configuration-document

It makes sense to split packaging a website from providing offline
storage/caching technologies, since the two are different solutions to
different problems.  It's true that there is some crossover,
especially if a widget includes content in its package.  But the
widgets spec has its own problems, not least of which is a terrible
name.

Letting users know that your site will work when offline doesn't seem
like a terribly difficult problem to solve at the application level,
to be honest. I think as a developer I'd rather browser vendors spend
their time on other stuff :-).  But it's certainly a fair point, and
does need to be considered as part of the UX of your app.

After thinking about it, he was right about separating out the packaging of an app from its offline storage capability. However, what was this widgets spec he mentioned? I’d never heard of W3C Widgets, so I decided to some research.

W3C Widgets are essentially packaged web apps. In fact, at one point, ppk proposed that they be put under the buzz word “HTML5 apps”. The W3C Widget spec discusses a format similar to Chrome extensions. You have a zip file of a directory structure containing HTML, CSS and JavaScirpt files, with a config.xml file that explains the basic metadata of the widget. Widgets can be embedded within a webpage or run from a user’s desktop. Not exactly what I was looking for, but it sounds kind of neat. Why had I never heard of this before? I did some more googling and the vast majority of the blogs and news regarding W3C Widgets seemed to be from 2009 or 2010. I could find nothing on Chrome, FireFox or IE implementing the spec – even though as of September 27, 2011, it became an official W3C recommendation.

I did, however, find a hand full of projects that used the spec. While there wasn’t a lot of buzz around W3C Widgets, there appeared to be enough going on with them that they weren’t in danger of falling into the abyss, at least not yet. Below is a list of the projects I found with some notes I made on them.

  • Opera’s Web Browser – Opera appears to have fully embraced W3C Widgets and is using the spec for Opera extensions. They’ve also set the browser up so that it will run widgets that you launch from your desktop. So its basically allowing you to create native-ish HTML5 apps. I find it peculiar that no other vendor has embraced W3C Widgets like Opera has.
  • Apache Wookie – This appears to be a widget environment and it reminds me of the OZone Widget Framework. Though Apache Wookie runs W3C widgets while OZone runs OZone widgets.
  • Phonegap – This framework allows developers to create HTML5 apps that can be distributed in mobile phone app stores. Its app configuration is based on the W3C Widget spec.
  • Wholesale Applications Community (WAC) – “An allegiance of telecommunications firms and others working together to create a common mobile platform” [ref]. W3C Widgets are apart of what they’re doing. However, according to this article, they ended up failing.

So each of these projects is kind of cool, but none of them addressed the original problem I had – how the hell are users supposed to know they have web apps installed on their computers that will work when they’re offline? And if W3C Widgets are the answer, why aren’t the major vendors adopting them? Eventually I had some nice people direct me towards the Chrome Web Store, and it was also nice to find out that FireFox will soon have its own web store too. Hopefully IE10 will do something similar when it comes out.

Even though I didn’t find them useful, it was interesting to learn about the existence of W3C Widgets. I’m not sure why they seem to have failed, as they’re essentially a way of packaging HTML5 apps. Though then again, I do think Chrome’s manifest is a lot nicer than the config.xml that the W3C Widget spec detailed.

Development Thoughts, General News, Web Apps

Notes on Offline Web Storage for TAAG

1 Comment

After you’ve visited it at least once, the Text to ASCII Art Generator (TAAG) will now work even when you don’t have an internet connection. I had a user request this, so I figured I’d add it in for them and anyone else out there who may want to use the app offline.

Offline web applications are one of the new capabilities being introduced with HTML5. An offline web application has its files downloaded and permanently installed in a web browser’s application cache. This allows users to go to an app’s URL and have it work even when they aren’t online. It’s a smart idea, but it has some interesting pros and cons. In this entry I’ll detail what I did to set TAAG up to work offline, detail some issues I ran into, and provide some notes for anyone looking create an offline app.

The first step in creating an offline web application is to create a manifest file which tells the browser which files it should store in its application cache, which files should try getting from the network, and if offline, which files it should use in place of certain standard files. Any HTML files that reference this manifest will be implicitly added to the CACHE section of the manifest and after a user’s first visit, all cached data will come from the application cache, even if a user F5’s the page. The application cache is only ever updated when the manifest file is updated, and even then, it will take 2 visits for a user to see any updates. This is because when a user visits the app’s URL, the application will be read from the cache, it will then see the manifest has been updated, and it will then update the application cache for the user’s next visit. These idiosyncrasies are a bit to take in, but once you know what’s going on its not that bad. Here’s what a sample manifest file would look like:

CACHE MANIFEST
# This is a comment.
# The cache is only ever refreshed when changed.
# Last updated: 2012.08.18

# The CACHE section indicates what we want cached.
# The HTML file linking to this manifest to auto-added to this list.
CACHE:
one.jpg
two.jpg
jquery.js?12345

# The NETWORK section indicates which non-cached files should be 
# obtained from the network. This will almost always be "*".
NETWORK:
*

# The fallback section allows us to setup "fallback" pages to use 
# when offline.
FALLBACK:
page2.htm page2-is-offline.htm

An HTML file linking to this manifest might look like this:

<html manifest="my.appcache">
<head><title>Test</title><meta charset="utf-8"></head>
<body>
<div>Some text</div>
<img src="one.jpg"/>
<img src="two.jpg"/>
<script src="jquery.js?12345"></script>
</body>
</html>

And one last gotcha – to keep the manifest file from being cached by the browser, and to ensure the manifest is served with the correct mime type (an incorrect mime type will keep the manifest from being recognized), we need to update the web server’s configuration. For Apache, we’d add the following rules to our .htaccess file:

AddType text/cache-manifest .appcache

<Files *.appcache>
    ExpiresActive On
    ExpiresDefault "access"
</Files>

That should do it! With this technique your users will now be able to use your apps even when they don’t have an internet connection. I find this cool because it makes web apps more useful and puts them a step closer to supplanting Desktop apps. However, there are unfortunately a number of issues that have held offline web apps back, and they’re worth mentioning.

Issues with application caching

A lot of people aren’t happy with how application caching currently works. The W3C has set up a working group that’s discussing possible improvements, though hopefully the main components of the current spec will continue to work. However, I also feel like they need to make some changes too. Below I’ll list my biggest concerns.

  • How would users know that certain URLs still work when they’re offline?

    I don’t understand how this isn’t the most important issue for offline applications. Without someone manually telling you that you can use a particular application while offline, I don’t see how someone would think they could browse to example.com/webapp and expect it to work when they didn’t have an internet connection. In their current form, I only see offline applications as useful to techies or people in the know.

    It’d be nice if offline applications could provide an icon, name, and description, and then browsers could show users which offline applications they had installed. It could be argued that this is a browser feature, but bookmarks are a browser feature too, and all browsers allow webpages to provide a favicon for bookmarking purposes. Whatever the case, there should be a way to let users know certain web pages/apps work offline.

    I’ve only seen this discussed once, and the people involved didn’t come to any conclusions, so I’m not too hopeful on this one. However, I think it’s probably the most important point, since no users = no usage.

    2012.09.01 Edit: Someone alerted me that there is a Widgets spec that’s currently in the works that supposed to solve this issue. I haven’t taken a good look through it yet, but it’s good to know there are attempts to solve this problem currently in the works.

  • Fresh updates aren’t served immediately / F5 doesn’t break the cache / the page that links to the manifest has to be cached.

    This seems to be the most popular feature request, though after reading spec author Hixie’s take on the issue, I sort of agree that many of the use-cases for using the application cache in this way are very similar to straight up HTTP caching. Though it would be nice if an app used the HTTP cache while online and this cache synced with the application cache, which would then be used when a user was offline.

    Around two months ago, a new feature was added to spec in an attempt to address this. It takes the form of a new setting that gets added to the manifest:

    SETTINGS:
prefer-online

    It’s explained here, though there’s some debate on if its the right solution, and I was unable to get it to work in Chrome or the nightly FireFox build, so I’m not sure how it works or if it actually solves the problem.

Other Notes

  • Chrome was easier to work with than FireFox

    Chrome will print to the console as its loading the application into the cache, so you know instantly if there’s a problem or not. FireFox didn’t do this and that made it a bit more annoying to work with.

  • Listing installed applications

    • For Chrome, browse to the URL “chrome://appcache-internals/”
    • For FireFox, in the menu, go to: “Tools>Options>Advanced>Network” and see the section on “Offline Web Content and User Data”.

  • In the manifest, URLs need to be URL encoded and query strings needed to be included

    Kind of obvious, but worth mentioning.

  • The popular iframe trick doesn’t work in FireFox

    There is a popular trick that aims to allow an HTML page to use the application cache while not being cached itself. It attempts this by having the page include an iframe with an HTML tag that links to the manifest. From my own tests, this trick works in Chrome but not in FireFox.

    2012.09.10 Update: After updating to FireFox 15, it seems to work now.

  • JavaScript API

    I didn’t go into it here, but there’s also a JavaScript API for the application cache. See the “Additional Resources” section below for more information.

  • What would happen if every website wanted to work offline?

    The space on a user’s computer is finite. I’m not sure how offline storage will work if offline apps end up becoming very popular in the future. Maybe the solution is to focus on installable apps which have the option of working offline – and users can pick and choose what they want installed?

Additional Resources

Development Thoughts, JavaScript

Tone Playing Experiment with HTML5’s Web Audio API

8 Comments

tl;dr: Tone Player (works in Google Chrome and Safari 6+ only)

One of my favorite parts of the QBasic class I had in high school was discovering how to play sounds. It introduced a whole new layer on which to experiment with.

Until recently, adding custom sounds to a web application has not been a simple task. Thankfully, the W3C is working on a new high level audio specification called the Web Audio API that allows you to easily create and play sounds. Webkit based browsers have pushed out an implementation of this spec (Chrome and Safari 6+), and hopefully other browsers will follow suit soon. Previously, Mozilla had been working on a more lower level audio API called the Audio Data API, but it is now deprecated.

After seeing a few fancy demos of the Web Audio API, I decided to do some digging to see what all it could do. Sadly, there aren’t a lot of great up-to-date tutorials on how to use it. A couple of exceptions I found to this were the getting started tutorial at creativejs.com and a hand full of nice blog posts by Stuart Memo. However, I was also surprised to find that the Web Audio WC3 specification is actually very readable.

So with this fancy new API, how hard is it to create and play a simple tone? It’s pretty easy actually. The below code gives one example.

var context = new webkitAudioContext(),//webkit browsers only
    oscillator = context.createOscillator();

oscillator.type = 0; // sine wave
oscillator.frequency.value = 2000;
oscillator.connect(context.destination);
oscillator.noteOn && oscillator.noteOn(0);

You can also play more than one frequency at once. However, I found that playing too many tones together can crash the browser, so this is something you have to be careful about, and there may be a better way to do this.

var context = new webkitAudioContext(),
    oscillators = [], num = 5, ii;

for (ii = 0; ii < num; ii++) {
    oscillators[ii] = context.createOscillator();
    oscillators[ii].type = 0;
    oscillators[ii].frequency.value = 2000 + ii * 200;
}

var connectIt = function(ii) {
    oscillators[ii].connect(context.destination);
    oscillators[ii].noteOn && oscillators[ii].noteOn(0);
    ii++;
    if (ii < num) {
        setTimeout(function() {connectIt(ii);}, 1000);
    } else {
        setTimeout(function() {
            for (var jj = 0; jj < num; jj++) {
                oscillators[jj].disconnect();
            }
        }, 1000);
    }
}

connectIt(0);

Oh, and you'll notice I check for the noteOn property before trying to use it. The spec says you need to use it, while Chrome tells me it doesn't exist. This appears to be a possible bug in Chrome's implementation.

With the basics down, I decided to create a simple Tone Player. It doesn't add much to what I've already showed you, but it has a few extra features. And in all honestly, it was actually made more to serve as a hearing test for myself. A couple years ago I had a hearing test done which indicated I had hearing issues, especially in my left ear. Since then I've been trying to take care of my hearing as best as possible. This tone generator is fascinating for me because I actually found that I can't hear certain frequencies (near 12000Hz), unless I really jack up the volume, but I can hear others fine all the way up to the 20k's. Hrm, probably time to schedule another appointment.

Anyway, enough about my hearing issues. Even though the API currently only works in a couple of browsers, it's a lot of fun, and I've hardly even dared to scratch the surface with this entry. I recommend checking it out if you have a few moments. If you're looking for a less experimental, more cross-browser way to play sound, I'd recommend checking out one of the many JavaScript libraries that have popped up to play audio. I'll link a few below for those who are interested.

Development Thoughts, General News

WordPress Hacking

6 Comments

A couple weeks ago I took an afternoon to read up on how to create a WordPress theme, and was surprised to learn how much there was to it. Re-doing this site’s theme has been on my TODO list for about 3 years though, and I felt it was important to give the site a fresh coat of paint.

From a user’s perspective, I’ve always loved WordPress. It’s intuitive, has a great interface, and has all of the blogging features I could want. However, under the hood, I’d heard it was a mess. After poking around a bit, I didn’t really find anything that was discouraging, but I did find myself spending way too much time researching how to make minor adjustments. So rather than toil endlessly, I decided to take a different approach and took the popular Twenty Ten theme and made a bunch of modifications to it (most notiably mixing in some elements from the Responsive theme). This was actually pretty painless, and I’ll probably continue to make more modifications. If you’re thinking about creating a theme, it’s worth reading up on how to do it, but using an existing theme as a launching pad will make your life a lot easier. Anyway, I hope the new design is easier on the eyes, please let me know if you have any issues!

Thoughts on PHP

WordPress is powered by PHP, and as of late, I’ve noticed a hand full of articles deriding the language, and another hand full vigorously defending it. I don’t like to consider myself a language specific programmer, but I do a fair amount of PHP development at work, and also find myself reaching for it when I do non-work related projects.

As of late I’ve been wondering if I should dive deeper into PHP, or instead try to look into getting good at Python or Ruby. The inelegance and quirks of PHP are a big turn off, but the fact that it’s used for so much (WordPress, Wikimedia, and lots of other popular software), and is so convenient to write and deploy, makes quitting it hard to do. I’ve sort of been in this weird stalemate about where I want to go. So while it’s not my favorite language, I don’t dislike it enough to throw the baby out with the bath water – for now at least.

Development Thoughts, Web Apps

TAAG Reboot

2 Comments

The HTML Frameset tag is officially obsolete in HTML5. Since their inception, frames have drawn the ire of many web designers, though I’ve always had a soft spot for them. When I first learned how to use the frameset tag back in ’98, I thought it was the coolest thing ever. I was able to do all sorts of trickery with it, like creating an online midi player and storing data between page loads. When I heard people go on about how they sucked, I ultimately knew they were right, but I felt like they were throwing the baby out with the bathwater.

Times have changed though, and as HTML has evolved, the gap framesets filled has been replaced by better technologies. Knowing their fate was sealed, I decided to overhaul the one app I had here that was still using them – the Text to ASCII Art Generator (TAAG)*. The new version was written from scratch, but with the aim to keep the same look-and-feel. This time around I also wanted to implement the full FIGfont spec, which surprisingly has a lot to it.

The main part of the spec that I didn’t implement last time was the vertical layout rules. These rules allow font authors to “smush” letters together vertically. Below you can see an example done in the “Standard” font.

  _____ _   _ _   _ 
 |  ___| | | | \ | |
 | |_  | | | |  \| |
 |  _| | |_| | |\  |
 |_/ _|_\___/|_|_\_|
  | |_| | | | '_ \  
  |  _| |_| | | | | 
  |_|  \__,_|_| |_|

This can be fun to play around with, but it’s not always what you want. Therefore, I decided to also allow users to control which horizontal and vertical layouts they wanted a font to use. Exploring what layout looks best is actually kind of fun.

The other main part of the spec I left out last time was the loading of non-standard letters. Most fonts don’t implement characters outside the normal ASCII range, but the FIGlet spec allows authors to define whatever unicode character they want, and some fonts implement quite a few extra characters. For fun, I went ahead and added unicode character #3232 to a few of the fonts, so that people could make a FIGlet look of disapproval:

   _____)      _____)
  /_ ___/     /_ ___/
  / _ \       / _ \  
 | (_) |     | (_) | 
  \___/ _____ \___/  
       |_____|

I’m probably too easily amused. Anyway, there are quite a few other updates, but if you’re interested you can read about them here.

I’ve also decided to open source the JavaScript FIGlet driver I wrote for this app. There were no other open source full-spec implementations of FIGlet that I could find, so I figured it could be useful. If you’re interested, it can be found here or here.

If you have any suggestions for the app please let me know. Also, I haven’t forgotten about the Keyboard Layout Analyzer, so don’t think this reboot means I wont be adding any more to that.

Oh, and as a final side note, it was interesting to learn that frames can be a drag on the loading of page. The additional HTTP requests should make this obvious, but that combined with whatever else the browser has to do to accommodate them, makes for a non-optimal load time. And interestingly enough, even though the new app has a heavier page weight (190k vs 114k), it loads much, much quicker – though I should say that frames are definitely not the only factor to influencing this.

* It’s former name was “Text Ascii Art Generator”. I noticed some people calling it “Text to ASCII Art Generator” and thought that was a better name.

Development Thoughts, JavaScript

Finding and fixing JavaScript errors that no one tells you about

1 Comment

Earlier this month I saw an interesting article on logging client side JavaScript errors, and then another one that showed how to do it using Google Analytics. The idea was clever and easy to implement, so I decided to try out the Google Analytics version when I posted up my new Keyboard Layout Analyzer last week. When an error occurred, I’d log the file it happened in, the line number it happened on, the browser’s user agent string, and the error message itself. Below you can see a screen shot of the error log after 1 week.

Google Analytics Panel

I’m unsure what errors 2-5 are. There’s no associated file, and the error message is simply “Script error”. The Analyzer gets an average of 90 visitors a day, so these account for a small percentage of users, though it is a little peculiar. Error 1 is definitely a real error though. I had been checking the stats every day, so when it popped up I actually got a little excited.

Usually when I discover an error, it stings. Users don’t often tell you about errors, unless they really like the app or are feeling unusually nice. Often times they may not even know something’s wrong. If the app wont work correctly, they just move on to another page without another thought.

This time, I caught it right after it happened, so I was actually a little happy. The error was pretty easy to pinpoint. Luckily it only occurred if AltGr keys were used, which doesn’t effect users unless they decide to add customizations for foreign layouts, which is why it didn’t show up until later in the week. I should have caught this while testing, but over sights are made sometimes.

So far I’m happy with this little experiment, and am thinking about possibly adding client side error logging to my other apps. For now the only draw backs I see are that this method of discovering errors can be a hassle if your scripts are minified and concatenated, and that users could abuse this if they knew you were doing it (though then again, they could abuse any kind of data saving request you were doing).

Development Thoughts

Testing your Linux skills

Photo By mkrigsman

If you’re a developer, you probably find yourself using Linux at least some of the time. Knowing your way around the OS can make your life a lot easier, and I’ve sometimes found myself wishing I knew certain commands or concepts a lot sooner. I’m no expert, but since I thought it might be useful, I’ve put together a short quiz to test your Linux skills.

Important Note: Since you are reading this through a feed reader or by email, the answers will be right below the question. On the main blog entry the answers are hidden until you mouse over or click on the “+” next to the question. Click here to be taken to the blog entry if you wish to not see the answers.

So, let’s get started…

  1. What does the cd command do when handed each of the following inputs: “-“, “/tmp” and “~”? [+]

    • “cd -” changes the current directory to the directory you were previously in.
    • “cd /tmp” changes the current directory to “/tmp”
    • “cd ~” changes the current directory to your home directory.

  2. What are two ways of copying a text file without using the cp command? [+]

    • cat the file and redirect the output to a new file.
    • Use the scp command (secure copy).

  3. You want your console window to continuously show you the latest updates to a log file as its being updated. How would you do this? [+]

    • Use the tail command with the -f flag.
    • Use the less command with the +F option (personal favorite).

  4. What is the difference between the “which” and “whereis” commands? [+]

    • which shows the full path of a given command.
    • whereis locates source, binary, and manual files for a given command.

  5. How would you find all of the files modified within the last day under a particular directory structure? [+]

    • find ./ -type f -mtime -1

  6. What is the purpose of the “/etc” and “/var” directories? [+]

    • The main directory layout for all Linux systems is based on the Filesystem Hierarchy Standard (FHS).
    • The “/etc” directory contains host-specific, system-wide configuration files (sometimes called “Editable Text Configuration”).
    • The “/var” directory contains “variable files”, or files whose content is expected to continuously change during the normal operation of the system (ex: log files).

  7. What are “cron jobs”? [+]

    • Linux systems have a program called “cron” which runs in the background and executes jobs (commands, scripts, etc) based on a schedule defined in a crontab file. This allows you to routinely do certain tasks. A “cron job” refers to a job run by the cron daemon.

  8. What does the “nohup” command do? [+]

    • Short for No Hang Up, this command allow you to execute a command that will keep going after you’ve logged out (ie, one that ignores hang up signals).

  9. You have a directory tree which contains all of the files for a web application. You want to make a list of all of the files that contain a phone number in the format of “(XXX) XXX-XXXX”. How would you do this? [+]

    • The key insight is to use the grep command with the -R flag (recursive directory search), -l flag (list files) and a regular expression to find the phone numbers.
    • As a side note, this used to be a popular interview question that Amazon gave to potential applicants (for more info, see the “Notes” section at the bottom of this post).

  10. What is the name of the official Linux kernel mascot? [+]

    • Tux.

Mouse over or click the +’s to see possible answers (though I didn’t include all possible answers – so you may have thought of some solutions that aren’t listed).

If you found yourself doing poorly, or just want to refresh your linux skill set, check out:

http://www.funtoo.org/wiki/Linux_Fundamentals,_Part_1

They have some of the best tutorials I’ve seen on Linux. However, if anyone knows of any other good tutorials let me know and I’ll add them here too.

Notes:

  • For more information on question 9, click here and go to the section titled “Area Number Three: Scripting and Regular Expressions”.