Book Review: Audio Processing with Web Audio

audio_book

Audio Processing with Web Audio

HTML5′s web audio API is pretty cool. Last year I had some fun experimenting with it, and since then I have been meaning to loop back and play around with it some more. So when I was offered a review copy of Audio Processing with Web Audio, my interested was piqued and I decided to see what it had to offer.

The book is setup to be a primer for web audio (in Packt terms an “instant” book). It contains a mere 63 pages, and reads like one long chapter. Overall it turned out to be a nice introductory book, but it’s does have a few flaws that take it down a notch.

As the book starts, you’re directed to setup your development environment and to download a zip of the code examples. This zip contains a total of 8 examples, broken out into 6 “recipes”. You can see a list of what they are below.

Recipe 1_1: Initialization exercise
Recipe 2_1: Playing Audio Files from an Audio Buffer
Recipe 3_1: Playing Audio in a Loop
Recipe 4_1a: Setting Volume
Recipe 4_1b: Setting Volume (refactored to use new template code)
Recipe 5_1: Scheduling Audio Playback
Recipe 5_2: Automating Audio Parameters
Recipe 6_1: 5-band Equalizer

For what I assume were space restrictions, the first two recipes in the code example packet were not discussed in the book. This makes the book slightly awkward since the first recipe covered, the one on looping sound (3_1), makes references to text written for these recipes and doesn’t cover concepts that would have been discussed for those examples. Thankfully it’s easy to pick up what you miss from the example code, but it’s an unfortunate editing snafu.

In the looping sound section you’re walked through the creation of the recipe via code samples with explanations, and once it’s done you’re given a deeper discussion of why things worked the way they did. The next two recipes are done in the same vein, and cover volume and automating audio parameters. The example code works smoothly and the author does a great job of explaining all of the little nuances that go with the code.

The final recipe is on building the 5-band equalizer. It’s billed as an advanced example, and it sort of feels like you go from a nice walk through the world of web audio, to a brisk sprint. I really didn’t know what was going on. I don’t know much about sound theory, so the different filters and concepts were all very foreign. I imagine this chapter was included so those more familiar with sound concepts would feel they got something out of the book, but it’s a little confusing for developers not tuned into how sound works.

Looking over the example packet, one other example titled “Scheduling Audio Playback” was cut from the book. However, unlike the first two examples, this exclusion isn’t really noticeable. Still, it leaves me thinking this book could have been a lot more interesting if it had been expanded by 20 or 30 pages. However, I’m glad they included all of the examples in the code packet, since it gives the reader some jumping off points after they’ve finished the text.

Overall it’s a nice introductory book. It has some flaws, but it comes with some good example code and does a good job on the majority of topics it tackles.

Random Updates for October

Over the last month I’ve done a lot of little updates. Now that the government is shutdown and I’m just hanging out at home, I figured I no longer had an excuse not to write them up.

If you let them build it, they will build it

I added the ability for users to submit presets to the Keyboard Layout Analyzer and the feature ended up being reasonably popular, with users creating and submitting lots of neat configurations. However, I was lazy in implementing the feature, and made it only partially automatic. Once a layout is submitted, I’m emailed the code I need to add to the config.htm file. This works out fine, since I only get about one or two submissions a week.

Since that feature worked out great, I decided to add a similar feature to the Text Color Fader, except instead of layouts, it was color presets… and oh did my laziness come back to bite me. It was originally setup to email me the code I needed to add after each submission, but after a week of getting 20-30 emails a day (and discovering that most submissions were just people testing out the submission feature), I realized I’d made a grave mistake. I went back and re-wrote it to include a proper admin panel and submission database. Now I just get a few emails a day and can easily approve/disapprove submissions. This approach definitely doesn’t scale, and with the number of presets growing daily, neither does the combobox I’m using on the front-end. However, I feel like I’ve bought myself time, and at the least there’s a neat color preset database that’s being compiled (users have submitted some very nice presets).

Ergodox Layout

When I originally overhauled the Keyboard Layout Analyzer, I put in support for different keyboard configurations – however, I never really showed this feature off. After getting a request to support the Ergodox layout, I knew I had an opportunity. I created a “key map” for it and uploaded it into the app. Within days users had submitted a number of nice alternative layouts for it, which was really cool to see.

The Years We Spend on YouTube

I read an interesting fact the other day. It stated that if you added up all of the views of Psy’s Gangnam Style, it would equate to over 13,000 years of human viewing time. I thought it was a pretty trippy concept, and since I’d sort of been looking for a reason to play with YouTube’s API, I thought it might be kind of cool to make a quick app around the idea.

You can see it in action here – Years Spent Watching YouTube.

Let me know if you have any suggestions for example videos.

Keyboard Layout Analyzer 3.0 Beta Release

I’m not completely done, there are still a hand full of new features I want to add, but I have a feeling that if I put off the update any longer, it’s never going to get posted up. So today I’m releasing the newest version of my Keyboard Layout Analyzer (KLA), which I’m dubbing version 3.0, since it’s the 3rd overhaul of the layout.

kla

New features:

  • Analysis results can be shared.
  • Charts are interactive / contain hover effects.
  • There are more “Display” options in the various results sections.
  • You can turn on/off certain keyboard data in the individual results sections.
  • The heatmap uses a more realistic heatmap visualization, has a table of key stats, and I’ve re-added in the mouse over effect (the biggest complaint in my previous version was that I took this out).
  • Miscellaneous section has charts for each piece of data.
  • Layouts you create can be submitted.

The biggest change though is the UI. The main reason I created this version was because I thought it would be a fun project to get better familiar with AngularJS. MV* JavaScript libraries like Angular have taken off recently because they make maintaining, updating and reusing front-end code easier. I felt the code I wrote for version 2.0 of KLA got a little spaghetti-ish at points, so this really appealed to me.

To keep things short, I’ll go ahead and just cut to the chase: Angular lived up to its hype. It was great at giving structure to the app. The new KLA code base feels a lot more sane and the HTML is easy to follow. If you do front-end development, I’d highly recommend you try Angular out. If you need a starting point, egghead.io has some great tutorial videos and Dan Wahlin has a really nice AngularJS Fundamentals video. And as a side note, even though Angular’s official documentation is good, I wouldn’t start there. I found it to be somewhat cryptic when I first started out. It’s a good reference point after you’ve gotten your feet wet though.

The only negative I came across with Angular was that it had a steep learning curve. I ended up having to back track a few times and re-do sections of the app because I didn’t do them the “Angular way”. Angular forces you re-think how to do certain actions and it uses some strange terminology, but once you get past all of that, it’s a pretty powerful tool.

Removing JK.js

My early days in JavaScript reminded me of an old puzzle game called The 7th Guest. In The 7th Guest, if you had trouble solving one of the puzzles, you could ask for a hint from the library. After getting 3 hints, the puzzle would be solved for you, but you’d miss the video that was shown after completing the puzzle, therefore missing a part of the game’s story. If you missed too many videos, you’d have no idea what was going on.

When I first started learning JavaScript, I kind of felt this way about third party libraries. I didn’t mind using ones that provided UI components, but I felt I’d be missing out on something if I used jQuery. So I wrote my own jQuery-like utility library called JK.js, which I used in my personal projects. However, I no longer feel like I’d be missing out on anything by implementing low level utility functions. What I really care about is writing an app, not a library. So as I’ve been refactoring my projects, I’ve been removing JK.js and replacing it with jQuery.

Flatly

Bootstrap makes it easy to create nice looking web pages. I decided to use an awesome Bootstrap theme someone created called Flatly, though I tweaked the colors a little bit. I also used an image from subtlepatterns.com as the background to make it feel a little livelier. However, while Flat is cool looking, I worry the result may be a little too flat, so at some point I may backtrack.

jqPlot

The charts in my previous KLA versions used a modified version of PlotKit. PlotKit was a good library, but it hasn’t been maintained in years. On top of that, my modifications to it used my own JS library JK.js, which I wanted to weed out, so I knew I needed a new chart library if I was to redesign this app. I decided upon jqPlot since it was jQuery-based and had a lot of great functionality. So far I’m pretty happy with it.

Heatmaps

I came across a neat visualization library for heatmaps, so I decided to use that over the simple key-color heat maps I was creating before. Interestingly, the author of this library also has his own keyboard heatmap app. When I saw this I felt a little weird about using the library – but since the heatmap library was its own separate component, I figured it wouldn’t be too weird if I used it too.

Future Updates

I still a have more features I want to add, ones that will be more stat based, so hopefully there’ll be another update in the next 2 weeks or so. I also may update my Typing Speed Test to use this same type of layout (it’s in dire need of an update). If you have any feature requests let me know. Special thanks go out to Wayne D. for submitting a bunch of awesome ideas and feed back!

Random updates for June

I feel like I work on this site a lot, but hardly ever post anything. Some of that may be due to me starting projects and then never finishing them though :P. Below is a collection of random thoughts and updates related to the site.

Gradient Image Generator

I’ve updated the Gradient Image Generator app and added support for CSS3. This update is in the style of what I was doing in my last post. I was going to do a “part 2″ post, but I changed how I was updating apps when I got to the Keyboard Layout Analyzer.

Keyboard Layout Analyzer Overhaul

I’m currently overhauling the Keyboard Layout Analyzer. This was long overdue since the chart library it uses, Plotkit, is long dead (in terms of active development), as is Mochikit, the JS library Plotkit uses. The new front-end will be powered by AngularJS and use jqPlot for its charts. I tried to find a place for xkcd charts, but couldn’t come up with anything that didn’t seem totally out of place. I’ll go into more detail on the new front-end when its finally up, and I’m open to feature suggestions if anyone has any.

TAAG

I added support for MySQL comments in TAAG. I also removed its offline web app capability. Offline web apps seem to confuse users, and I’ve noticed downloading errors in Chrome while using it on my iMac. This seems to happen randomly and because there are lots of files that need to be cached, and it doesn’t seem to be repeatable. With the appcache API being in disarray, and after running into bugs in Chrome and FireFox, I’ve lost my enthusiasm for this particular web browser feature.

iMac

I’ve forsaken my Microsoft roots and purchased a souped-up, 27 inch iMac. I wasn’t a fan of Windows 8, the wires from my PC were making me nuts, and I thought it might be interesting to switch things up. I’ve only had the computer for a few months, but so far I’m pretty impressed. The big screen is amazing, I love being able to use terminals, updating software with brew is nice, and I’m a big fan of the magic mouse. The only downside so far has been that I can’t run Internet Explorer. I’m not sure how Mac users did web development 5 years ago…

Google Authorship

I feel like I’m the last person to know about this, but figured I’d mention it here anyway. Have you noticed people’s faces showing up next to certain search results? Apparently it’s called Google Authorship, and it’ll let you stick your picture next to webpages that you author. It only seems useful for blogs and news articles, but it’s kind of neat.

Spam Comments

When I get a blog comment that I think is spam, I google it to see if its been posted elsewhere. Usually my gut is right, and I see the comment posted on a dozen or so other blogs. Recently though, I came across a spam bot that had messed up and posted this:

{
{I have|I've} been {surfing|browsing} online more than {three|3|2|4} hours today, yet I never found any interesting article like yours. {It's|It is} pretty worth enough for
me. {In my opinion|Personally|In my view}, if
all {webmasters|site owners|website owners|web owners} and bloggers made good content
as you did, the {internet|net|web} will be {much more|a lot more} useful than ever before.
|
I {couldn't|could not} {resist|refrain from} commenting. {Very well|Perfectly|Well|Exceptionally well} written!|
{I will|I'll} {right away|immediately} {take hold of|grab|clutch|grasp|seize|snatch}
your {rss|rss feed} as I {can not|can't} {in finding|find|to find} your {email|e-mail} subscription {link|hyperlink} or {newsletter|e-newsletter} service. Do {you have|you've} any?
{Please|Kindly} {allow|permit|let} me {realize|recognize|understand|recognise|know} {so that|in order that} I {may just|may|could} subscribe.
...
}

As you can see from its setup, it appears to randomly choose synonyms for many of the words. It also has around 2 dozen comment templates (for the sake of brevity, I’ve only included the first two). I was actually kind of impressed, that’s pretty clever! They knew how people were checking for spam, and adapted to try and get around it. Pretty soon spammers will be using AI to analyze a blog’s content, and then use that information to post a relevant comment or follow-up comment, and bloggers will have no idea that they’re conversing with spambots.

Design Overhaul (part 1)

Like a book being judged by its cover, people tend to judge an app by its UI. Since jumping into a web development job 3 years ago, I’ve done a fair amount of client side work. I’ve helped redesign legacy applications with cool new UIs to lots of praise, even though the new apps had less features. And I’ve seen customers balk at feature-rich products that had so-so interfaces. And it makes sense, people like to use stuff that looks good, and if you can’t be bothered to put work into the design, what does that mean for the rest of the app?

I’m not a designer, and most of the stuff on this site was created before I really knew much about design. So as a result, a lot of what’s here looks pretty meh. After writing my last post, I took a look at the text fader that’s on this site and was kind of embarrised. I decided something needed to be done, so I started going through the various sections of this site and giving the layouts a much needed fresh coat of paint.

Mobile Version

The new designs aren’t amazing by any means, but I think they’re a step up, and hopefully they’ll provide a better user experience. I decided to use Twitter Bootstrap for a lot of the basic look and feel, since it looks great and plays well with jQuery. I’ve also been trying to make the new layouts responsive, so that they also look good on mobile devices.

I’m not allowed to bring a cell phone into work, so until last year, I avoided getting a smart phone. However, now that I have an Android, I use it all the time outside of work, and I’ve realized that I’ve made a big mistake by avoiding mobile development.

So now that the background is out of the way, what has actually been updated so far?

There are several more sections of this site I’m going to overhaul, but figured I’d do a write-up of what’s been done so far. I’ll also probably go back an revisit some of the sections I’ve already done if I get some better ideas. If you have any suggestions for anything just let me know.

Was Mark Zuckerberg an AOL add-on developer?

mark_adjusted_350

Facebook founder Mark Zuckerberg’s first website was recently found to still be online at Angelfire, an early free web hosting site. The Internet Archive confirms the site existed in its current form back in 1999, and the page’s source code is noted to be authored by “Mark Zuckerberg”. In addition, the author states they’re 15 (the age Mark was in 1999), and that they live outside of New York City (where Mark lived when he was 15). Motherboard provides further evidence, showing that the primary AOL account for the email listed on the site is a name commonly used by Mark Zuckerberg’s father.

The site screams 1999 web design, and is a very cool piece of internet archaeology. It should also be noted that it’s actually a pretty decent effort for the time for a 15 year old (you only have to see the About section of this blog to see my effort at 16). However, the most interesting aspect of the site is “The Vader Fader”, an AOL add-on application that Zuck was heavily promoting on the site. Did this mean he was apart of the AOL add-on community? Did he use AOL progs? Did he develop in Visual Basic?

vader

I downloaded The Vader Fader, and it is for AOL, and it was indeed written in Visual Basic. I tried firing it up, but got a message box saying I needed to be “online” and then the window on the left popped up. Ugh, I just want to see what this app looks like, I have to have AOL open? So I hunted down a version of AOL 4.0, installed it, and then tried running the app again. This time I got a runtime 6 error – this was most likely caused by Zuck using Integers to store window handles instead of Longs. After Windows 98, window handles started being Longs instead of Integers.

Being persistent, I decided to download Windows Virtual PC and load up Windows 98. After burning AOL 4.0, The Vader Fader, my API Spy, and a hand full of VB dependences to CDs and then loading them up on the OS, I fired up The Vader Fader. This time it didn’t crash, but it still told me I needed to be online. Crap… how was I supposed to do that? I tried signing on, to see if by some fluke AOL was still active and letting random people sign in, but it didn’t work. It then occurred to me – how did progs back in the day determine if someone was signed on? I couldn’t believe I remembered this, but the way it worked was the app would find the main AOL window, and then look for a child window that had a caption that started with “Welcome, “.

I used my API Spy to change the caption of the existing AOL sub-windows to “Welcome, PAT or JK”, and then tried launching The Vader Fader again. This time it worked! Well, sort of. Instead of a message popping up, the caption of the main AOL window changed to “The Vader Fader”, and then nothing happened. I poked around, and the app was running in the background, but there was no main window and it didn’t appear to have done anything else. My best guess is the app worked by augmenting AOL chat rooms and IMs with fading options (why else would it change the main AOL window’s caption?). If that’s the case, there really wouldn’t be much to see, or really any way to see it – given that AOL 4.0 chatrooms and IMs are long defunct.

I was a little sad, but glad I’d at least gotten the app up and running. I also ended up digging through the app’s machine code a little for any other clues on how it was created, but didn’t really find anything interesting (other than the 10 color choices). Since the app used the same online detection mechanism as most other apps at the time, I wonder if Zuck used a common bas file like dos32.bas or genocide.bas – that’d be pretty cool if he did. It’s also kind of neat that the main app he was pushing was a fader, since that was the first app I released on this site. Anyway, I’ve spent way too much time on this. The site is a cool piece of internet archaeology and definitely worth poking around a bit if you have a few extra moments.

Facebook User Identification Bug

Time for Round 2

I decided to take another shot at Facebook’s Security Bug Bounty program. This time I ended up finding a bug that allowed a website to use Facebook to detect if their visitors were particular people. After doing a write-up and submitting it to Facebook’s security team, I was awarded a $1,000 bounty. Below I’ll go into how I came across the issue and how the technique worked. Also, I don’t see myself going full-on bug hunter after this or anything like that, this has mostly just been a random side-adventure that came from me being inspired by some random blogs I read on bug hunting and application security.

How it worked

I noticed that the preview-image for a Facebook Badge was dynamically generated based on a user’s Facebook ID. There were some other configuration parameters, but for simplification, you can imagine the HTML looking something like this:

<img src=”https://facebook.com/badge_edit.php?id=12345″ />

When rendered, the image would look something like Figure 1. Since it could contain a user’s email address, my first instinct was to see if I could load other users’ badge previews and get their email addresses. However, when I tried this, a Facebook logo image loaded instead of the profile image (Figure 2).

Figure 1

Figure 2

This meant that only the user themselves could view their profile badge preview. However, even though the logo was put in place to block information from being leaked, I realized that it still leaked information. An external website could use JavaScript to silently load tons of profile badge previews with different user IDs, and then check their height and width. If one of these images loaded and it wasn’t the height and width of the Facebook logo, the website would know that that particular user was viewing their page.

This had a lot of interesting use cases. Someone could:

  • See if a particular acquaintance (boss, ex-girl friend, frenemy) was stalking their Facebook feed or reading their blog.
  • See if someone famous was visiting their site.
  • Track the actions of certain people.
  • Show different content depending on who was visiting.

The one limitation was that someone could only check against a list of certain people, but there are only so many people someone can know, and targeted attacks are for particular people anyway.

Submitting the Bug

It was 1am Monday morning when I found this bug. When I realized what I could do with it, I had a rush of adrenaline and stayed up another 3 hours coding up a proof of concept and submitting a bug report. I really should have just gone to bed and wrote the report the following day (I’m supposed to be in at work by 9am), but it felt sort of like when you’re on the last level of a video game and your mind is buzzing with “holy crap, I’m going to beat the game!”

Similar to my last report, it took around two weeks for a security analyst to get back to me, and similar to last time, I was told it was interesting but that they needed to check with their engineering team. I was a little worried at this response – however, I felt that this time I’d found something that was really cool. Hell, it’s something that I would be tempted to use.

About a week later I noticed the bug was fixed and I received an email asking me to verify the fix. I did a few tests and then let them know that everything checked out. Shortly after this I got an email from the security team letting me know I was eligible for a $1,000 payout and that I needed to fill out a W-9.

Final Thoughts

In addition to Facebook, I also tried the white hat programs of a few other sites, and even ended up making Google’s honorable mention list and earning Reddit’s white hat profile badge. It’s kind of cool to get little bit of recognition and pocket change for finding security holes, though part of me is unsure if it’s actually worth it, since it is a decent amount of work for no guaranteed returned. However, overall I am glad I did it since I learned a little more about security and it felt great to score a few finds.

Crafty Tricks for Avoiding XSSI

While looking at the response data from certain AJAX requests in GMail, I noticed that they were prepending their response JSON with either a number, “while(1);”, or “)]}’”. They do this to avoid cross-site inclusion attacks (XSSI), where a malicious site includes a file from another site in order to steal its data. Example:

<-- Code on evil site -->
<script src="http://gmail.google.com/user_email.json"></script>

Even if the data in the JSON object isn’t wrapped in a function or set to anything, it can still possibly be stolen by overriding the Array constructor or defining a setter property on the Object prototype. Most of these holes were plugged up after being discovered, but they still exist in some older browsers, which is why GMail is putting these safe guards in place. Two of the techniques of the mentioned, “while(1);” and “)]}’”, are pretty straight forward. “while(1);” will cause an infinite loop, keeping an attacking site from reaching its payload, and “)]}’” will cause a syntax error, stopping the rest of the script from executing. But the number trick is not so straight forward, and it had me scratching my head. The response data looked something like this:

123
["Email message", 123, "?", 0];

Was this to cause a syntax error? I decided run the code – nope, no syntax error. In fact, it turns out that the above code is valid JavaScript. However, instead of returning the last line, the array, it returns undefined. Wait a second, how could that possibly evaluate to undefined?

To experiment more, I decided to try executing:

123
456;

Which evaluates to 456. And then:

123
"test";

Which evaluates to “test”.

So when the second item is an array, it evaluates to undefined, but when it’s a number or string, it evaluates to the second parameter? WTF JavaScript? After showing the issue to a few co-workers, none of us were able to figure out what was going on, so I took the question to StackOverflow.

Within a few minutes, freakish had the answer. It turns out that JavaScript’s automatic semi-colon insertion was to blame. In the second and third cases, JavaScript realized that “123 456″ and “123 ‘test’” weren’t valid statements, so it inserted a semi-colon at the line breaks. However:

123 ["Email message", 123, "?", 0]

IS a valid statement. What’s happening is that the []‘s cause JavaScript to look for a property value on the number object. The comma statement returns its last parameter, 0, so JavaScript looks for a property named 0 in the number’s prototype chain. When it finds nothing, it returns undefined. This can be tested with the following code:

Number.prototype["0"] = "test";
123["Email message", 123, "?", 0]

Which evaluates to “test”.

Wow. That is so completely ridiculous. However, it’s also pretty clever on Google’s part. It allows them to have another XSSI-busting technique, and I’m assuming they use a variety of methods in case a hack is discovered which gets around one of the other techniques. Anyway, I found the number trick to be especially interesting, so I figured I’d share it here. And again, freakish deserves the credit for figuring out what was going on.

The Chrome Web Store Effect

It’s been about 5 months since I tried out Chrome’s Web Store for my Snake and Text to ASCII Art apps, and I figured I’d give an update on how putting them in Chrome’s Web Store has effected their usage.

My Snake game has sat around for about 4 years on this site, consistently getting an average of around 30-50 users a day. I originally wrote the game as a nostalgic tribute to my high school days, and it’s only undergone a hand full of updates since its initial inception. Since this site was getting around 6.5k visits a day, 50 visitors means the app was accounting for around 0.077% of the site’s total traffic. After placing the app in Chrome’s Web Store, the daily usage began to steady rise, and the app has recently been getting between 900-1,000 visitors a day on the weekdays:

Snake Stats

Holy crap, the Chrome Web Store is awesome! Even with the bagillion other Snake games listed in the store, traffic has gone through the roof, with daily usage up 2,000%! But what about the TAAG app, has it also seen such a meteoric rise in traffic? Interestingly, even though both the Snake app and TAAG app have a similar number of installs, TAAG hasn’t seen a noticeable rise in traffic:

TAAG Stats

Hrm, this is kind of interesting. My guess is that this discrepancy is caused by the following factors:

  • TAAG is a utility app and isn’t really something that’s used often.
  • There is a high volume of people trying out games in the Web Store (installing them, and then uninstalling them if they don’t like it).

It’s hard to be certain of what’s actually going on, but I found these results to be pretty interesting, so I figured I’d share. I had wondered if the Chrome Web Store would be a positive for web app developers, and so far, from my limited experience, it seems like could potentially be a big plus for web app developers, especially game developers.