All posts by patorjk

Development Thoughts, Introspective, JavaScript, Personal

Becoming the Maintainer of an Abandoned Open Source Project

7 Comments

Lots of ink will probably be spilled documenting the stories of what people did during the summer of 2020. With COVID-19 spreading throughout the world and many places going into lock-down, it was an uncertain and strange time. Many people took the opportunity to catch up on old TV shows, learn about investing, or create a side hustle. Me? Well, somehow I found myself becoming the maintainer of one of my favorite open source projects.

Part 1: The Big Rewrite

Almost 4 years ago I was gearing up to rewrite a rather huge AngularJS app. I had been a bit disillusioned with Google’s decision to kill off AngularJS (don’t get me started with Angular 2), and I was looking to build an app that wouldn’t be tied to the whims of Google, since they have a tendency to shut things down. After surveying the landscape, a solution involving the open source libraries React, Redux, and Material UI seemed like a good fit. All that was left was finding a good datatable component…

mui-datatables looked clean and worked smoothly. It made my app look beautiful, and I was excited. To add to this, the official github repo seemed to be buzzing with activity. My first feature request got 12 reactions and a bunch of comments. There was clear enthusiasm about this project, and it stood out amongst all of the Material UI based datatables I evaluated. I felt like I’d found the final piece to my puzzle.

For a while everything went smoothly. I was no rush and my work had me juggling several projects, so the rewrite took place in-between other tasks I was doing. It wasn’t until I was a few months in that I realized I needed an ability that mui-datatables didn’t provide. “No worries” I thought, “I’ll just put in another feature request”, but when I checked the repo, I noticed Greg, the library’s creator, was no where to be seen. Instead, someone named Gabriel was now running the show.

At first this was fine, and I was glad to see someone had taken up the mantle and was moving the library forward, but I soon found myself increasingly frustrated. I was so close to having the library that I needed, yet many of my pull requests (PRs) were left to languish and some updates to the library made my life harder. To add to this, Gabriel wanted to rewrite the whole library, deeming its internals fundamentally flawed. He made a pinned issue announcing this big rewrite (“Version 3.0”), and said it would fix the table’s internals and possibility completely change its API. However, after this announcement he continued to work on features for version 2.x, and in updates months after this announcement he would state that he hadn’t started on version 3 and was still thinking about things.

I was concerned. Once again I looked at other datatables, but I was filled with dread when I concluded that mui-datatables was still the best fit for what I was doing. I either had to stick it out and convince Gabriel to take my PRs, or fork the library and be on my way… and well, I forked it. And that actually turned out to be a pretty freeing experience. But there are downsides to forking that cannot be ignored:

  • You don’t get updates from the community – you’re on your own.
  • When you eventually leave your position, you leave the next developer with learning the fork you made. I’ve been in this boat and it’s not a fun boat to be in, especially as the fork gets old and bugs are found.

After the app I had been working on launched, I thought a little more about these 2 points. One of the key reasons for the rewrite (but not the only reason) was to get off of the now archaic AngularJS framework so that future developers could easily jump into the project. But leaving someone with a forked library that was heavily modified seemed to clash with this idea.

I decided to once again check in with mui-datatables to see if there was some way we could reconcile. What I found was 60 PRs, 2 pages of unanswered issues, and someone @-ing Greg and Gabriel, asking if the project was dead. The last release was 3 months old, and it had a number of problems (a malfunctioning resizable column feature, the responsive design was broken, etc etc). After Greg and Gabriel, I was the 3rd biggest contributor to the project, and with all the updates I’d done in my fork, I realized I was probably the most qualified person to take over. To add to this, COVID-19 was spreading around the country and my work had recently closed down. I was at home with nothing to do for the foreseeable future. I needed something to take my mind off things.

I decided to reach out to Greg on Twitter to let him know the situation and ask if I could take over. I’d previously sought him out to ask why he’d left the project so I knew he’d be responsive on Twitter. A week later he handed me the keys. I then went to 2 of my oldest PRs, merged them in, and did a release. I was the new maintainer.

Part 2: mui-datatables 3.0

Gabriel briefly returned to wish me well. He told me his time had become scarce, and he simply didn’t have enough of it to spend on the project. Even though I’d felt a lot of frustration with some of his decisions, we’d been friendly and I appreciated the work he’d done. Had he not taken on the project, it most likely would have died. However, a year had passed since he’d made his announcement about the next version of the library. Was a version 3 still on the table? I’d done a lot of work on my fork and there were a lot of PRs. Maybe there was enough new stuff to justify a big release.

For the next two weeks I poured through the PRs and open issues. It was oddly fun and proved to be an eye opening experience. Some of the stuff people submit is completely nonsensical while other things are highly complex and clearly had a huge amount of work put into them. For example, one person had rewritten the whole library in TypeScript, and while this was a neat idea, it was completely impractical and would have made it almost impossible to merge the other updates in (plus, I’m not completely sold on TypeScript – but that’s another story). On the opposite end, there were some requests that were small with little explanation. Often times they didn’t work at all and/or caused the tests to fail. It was like people submitted their work without checking it.

I tried to be nice. After all, each PR represented someone spending their own free time to better this library. At the very least they deserved someone trying out and reviewing the changes that they made – even if they had to wait a year. And to my delight, people were pretty cool. I either got a positive response thanking me for looking at their PR, or no response at all.

Of the 60, I ended up accepting 23. The vast majority of these were bug fixes and minor updates. The only submission that really fell into the “cool new feature” category was one that added an injectable component feature. During this period I also ported over most of the updates from my fork, which in the end, accounted for all but 5 of the new features/API updates. A thorough review of the code base was also done which cleaned up a hand full of anomalies. For example, most of the deprecation warnings had accidentally been disabled in version 2.14.0, and starting in version 2.13.1, a large 5MB file was accidentally being included in the npm package. No one seemed to have noticed these things though.

I also updated the library from using version 3 of Material UI to version 4. In the year that had passed, most of the Material UI community had moved on to version 4. Not updating the library to correctly work with version 4 had probably hurt adoption. When I had parted ways the previous year, mui-datatables had 15k downloads per week on npm and it’s closest competitor, the feature heavy material-table, had 7k a week. Now the tables had turned. Material-table was crushing it at 80k downloads a week to mui-datatable’s 25k. The version issue most definitely wasn’t the only reason mui-datatable had lost ground, but I have to imagine it was a significant factor in people’s decisions.

In the end, version 3 would be no great revelation, but it would be a step forward and hopefully a step in the right direction.

Part 3: The Rise and Fall of a Maintainer

After it was released a few people chimed in to say thanks and report bugs, but there was no big celebration that the library was back to getting updates. I got the impression that many people using the table had built their apps around it a year ago. It didn’t seem like it was attracting a ton of new users. From github stars, I could tell that on average, 1.35 new people were starring the library a day, which seemed a bit lower than it had been in the past.

To get things going again I decided to start work on two features I felt were essential:

  • A cell rendering method that would significantly boost performance.
  • Draggable columns.

I had a soft spot for draggable columns. It had been discussed with feverish enthusiasm when I was originally looking at the table, and I felt like it would be poetic as the first big feature of the 3.x era. I wanted something that was nice though, I didn’t want something cheap looking. So I got to work and created what I felt was a slick draggable column feature:

click to play or pause

Most people will say they don’t care about small little effects like this – that all they really want is the functionality, but over time I’ve found this to be false. Little touches like this add up, and overall lead to users liking a product more. 

As I prepped a new release, I began to talk about new features with the few of the people still hanging around. Maybe a grouping feature should be next? Editable columns? I was excited. I was going to get this table back on track and soon it would have features that rivaled material-table. But in addition to the lack of activity in the repo, something else bothered me: Surely I couldn’t be the only one who felt datatables for Material UI were lacking? Hell, when I was doing AngularJS there were several great community options. What was going on?

I went digging, and found my answer on the material-ui github repo. In a thread lambasting material-table, several people stated that they weren’t happy with the community options. The co-creator of Material UI, Olivier Tassinari, satiated the criticism by ensuring them that they were hard at work on an official datatable component. It would be ready for preview in September – basically at the end of the summer.

I had been out of the loop, and though it seemed obvious, apparently the community was displeased with both mui-datatables and material-table. The creators of Material UI realized they needed an official solution, so in October of 2019 they’d announced plans to create one. That explained why no other table had come forth, and it made me feel like mui-datatables and material-table were both lame ducks. A good solution was on the way, and there was no point in a community project if an official table was going to be supported. (however, I would later learn that certain parts of this official table were for paid users only – so material-table and mui-datatables would still have a place in the future)

I was a little distraught, but decided to continue work, albeit at a slower pace. Then, after 10 weeks of being at home, I got the call to come back to work. I reintegrated mui-datatables into work my project and showed off some of the new features. My team lead seemed impressed and was thankful we were no longer using a forked project. With work on mui-datatables now restricted to evenings and weekends, my contributions to it slowed even more. Then, one day in late September, I handed in my resignation at work.

Wait, what?? Oh yeah, probably forgot to mention that during my time off I was kind of stressed about my future. My work had been extremely generous in paying us to stay home and do nothing, but there were rumors about leave without pay in the Fall. With no telework option available and no assurances made about what might happen once the leaves started turning, I had decided it was best to hedge my bets and look for another position.

In a twist of fate my new job would involve working on another AngularJS 1.x project (it never ends!) and possibly porting it to React (though as of now that hasn’t happened – my guess is we’ll still be dealing with AngularJS 1.x apps 10 years from now, though that’s a topic for another time). Now I had even less motivation to continue work on mui-datatables. I didn’t want the project to fall back into disarray, so I felt like the only reasonable thing to do was to find a successor. Thankfully during my time as maintainer another developer, Woo Dohyeong, had joined me in my quest to better the library. He was the obvious choice to take over and with Greg’s blessing, I passed the torch. After Woo made his first release I stepped back.

It was bittersweet. Part of me knew I didn’t have enough time to be a maintainer forever. It’s a job that gobbles up the extra minutes of your day and its mostly thankless. I didn’t talk too much about it above, but there were a few days where I would handle half a dozen questions, and the majority of people wouldn’t say thanks or respond, some people would even be rude. However, reviving the table and improving upon what so many others had built was rewarding. There was a sadness in stepping back, but it was for the best.

Final Thoughts

Well, I didn’t expect this to be so long, but the story (even trimmed down) turned out to be much longer than I thought it would be. I needed a place to write it down though, and if you read it, thank you for reading my story. The summer of 2020 was a crazy time, and even this bloated blog post barely scratches its surface. Hell, I didn’t even write about my bike rides through empty streets or obsession with Hollow Knight (and Animal Crossing, and Cuphead), not that those things are in any way relevant to mui-datatables, but they filled the gaps between development. Anyway, hopefully you found this entertaining or enlightening. Next time you use a piece of open source, be sure to show appreciation to the maintainer, and don’t be afraid to contribute yourself. If you have the time it can be a fun little adventure. Also, don’t get too caught up in the endless cycle of front-end rewrites. There’s a certain madness to it.

Uncategorized

New patrickgillespie.com

7 Comments

2021 Update: I’ve since re-done the site again, when I get a chance I’ll write something up on it. The post below refers to an older design.

***

Another redesign, this time a good one, I swear! I decided to learn ReactJS, and for my first project I redesigned my photography website, patrickgillespie.com

My previous design was pretty bad, I guess no one had the heart to tell me. For this new one I studied several other designs and took elements I thought worked best. It’s simple, responsive, and the code is up on github if anyone is looking for a portfolio template.

patrickgillespie.com had been pretty dead for about a year, receiving an average of 2-3 visitors a week, probably bots. The domain was about to expire earlier this month, when I noticed a big uptick in visitors leading up to the expiration day (~20 a day). Were these other Patrick Gillespies eyeing the domain? Domain vultures looking to scoop up a site that was about to fall into the abyss? I guess I’ll never know.

patorjk.com, on the other hand, has been alive and kicking, at least visitor wise. No one reads this blog, but the apps on the site see hundreds or thousands of visitors a day, making me feel sort of bad about neglecting the site. It’s like a rudderless ship that has somehow managed to successfully sail the ocean.

Anyway, if you’re reading this, I hope you enjoy the new portfolio site or at least find some use in the code. The most useful piece of it is probably the create-image.js script, which creates multiple sizes of a photo and extracts it’s exif data into a JavaScript object which the application can use.

Web Apps

Using Dithering to Create Old School Gaming Filters

6 Comments

Recently I’ve been reading up on image dithering. It’s kind of cool. It’s a way of transforming an image into a smaller color space while still simulating as much color depth as possible. It has lots of practical uses – printing, displaying images on screens with limited colors, etc – but I realized it could also be used to inject images a heaping dose of gaming nostalgia. I couldn’t resist doing a quick proof of concept, so I’ve created a new web app that will transform an input image into what it may have looked like on an old gaming console by way of various dithering algorithms. For example, below you can see what I may have looked like on a Game Boy.

As you can probably infer from what I said previously, the app is actually pretty simple. All it’s doing is resizing the image and applying an image dithering algorithm to it.

There are various image dithering algorithms, but most work by trying to intelligently add noise to areas of an image where it doesn’t have the right colors in its reference palette. This noise can help simulate the missing colors. One place you may have seen image dithering is in gif files, which typically have small color palettes. In fact, the noise you see in many gif files is commonly due to the dithering algorithm that was applied to it. Without dithering, these files would suffer from color banding, which can be pretty ugly. In the video below you can view comparisons of what undithered vs dithered images look like given color palettes of various sizes.

As you can see, dithered images are able to look much better with far fewer colors. For the new app, I chose to use the popular Floyd-Steinberg dithering algorithm for most of the results (and used the wonderful RgbQuant.js library for this), and also used Ordered Dithering for a few other cases (I couldn’t find a good implementation of this algorithm, so I just wrote my own version based on what I read here). If you play around with the app, you’ll notice that the results with ordered dithering are much worse. I only included this option because it appears to be what the Game Boy Camera used. I’m not 100% sure on this, but the cross-hatch pattern found in many Game Boy Camera images is indicative of ordered dithering.

I should also note that while the new app can give you an idea of what an image may have looked like on an old gaming console, some of these filters wont be exactly 100% true to life. For example, the NES is capable of displaying around ~54 different colors, but for a single sprite it could only display 4 colors. So it wouldn’t really be realistic to expect an NES to display an image like this:

For it to display that image, it’d have to be composed of a bunch of tiny sprites that overlapped, and I’m not sure if there would be any limitations when doing that. So these filters are really just best guesses, and mostly just for fun.

Lastly, I’ve also put in an option to create output without dithering. In some cases this produces a more realistic gaming result, and sometimes combining dithered and non-dithered images can give a better result, like this Sonic image: (I combined the dithered and undithered versions in photoshop, masking in the areas I liked from each version).

Honestly I’m not sure what practical use this app can have other than maybe a few minutes of fun, but I hope you enjoy it. If you’re interested in learning more about dithering algorithms check out this blog post on the subject, it covers in detail how several of the more popular ones work.

  • Check the app out here: Old School Gaming Filters
  • Create your own filter here: Palette Swap (this was actually my original idea, but it takes a little more work and I found pre-made filters were a little easier to work with)
Uncategorized

New YouTube Channel

2 Comments

I’ve started a YouTube channel. So far I’ve only created a few videos, and I’m still trying to figure out what I want the channel to center around, though below you can see a couple of the videos I’ve created.

I figured the keyboard one might be of interest to some who visit this site, since it ties into the Keyboard Layout Analyzer app. The drone videos are fun, and there are a lot of neat places around me. Right now I’m trying to think of things the utilize photography and technology, though I’m still not sure what I want the channel to center around. Anyway, hope you like the videos.

Old School, Personal

Days become Weeks, Weeks become Months

1 Comment

It’s April. I’m now 34. I swear it was just yesterday I was eating lunch in the Commons at UMBC. Now I’m neck deep in adult stuff like figuring out my taxes and changing poopy diapers. People talk about how having 2 kids is more than twice as hard as 1 kid, but I think it’s just that the new kid takes up all the rest of the time that you did have, so you don’t really get time to rest.

patrickgillespie.com

I’ve finally decided to do something meaningful with patrickgillespie.com. I purchased it 8 years ago and have mostly neglected it. I previously had it setup as a “Portal for Patrick Gillespies”, but that was kind of a lame idea.

The other day I realized it would probably be a good place to show off my photography. Every photography youtube channel I watch says you need a portfolio to show off your best work. I have the domain, why not setup a portfolio? Therefore, I’ve recently re-launched the site as Patrick Gillespie Photography.

I wrote the site using AngularJS, Angular Material, and jmpress.js. I want to polish up the code a bit, but when I’m done I’ll put it up on github. This is my first time creating a portfolio, so I’m kind of just having fun with it. I’ll probably continue to add stuff to the site as time goes on.

AOLers Reconnecting

There’s a reasonably active Facebook group on the topic of old school AOL development that someone created a while back. If you were a member of that community it’s worth checking out.

Introspective, Personal

Where is Bob?

Leave a comment

You know you’ve reached a strange point in your life when you find yourself reminiscing about stuff you saw on Reddit almost a decade ago. However, sometimes something you read or see online just sticks with you. It resonates. Whether it makes you think, feel, or laugh, it leaves an impression. The “Where is Bob?” blog, which appeared on Reddit back in 2008, was such a thing for me.

Penned by a wickedly funny IT worker, the Where is Bob blog hosted a set of possibly non-fiction stories about a work place run by an obnoxious and often absentee manager named Bob. The submission garnered the praise of the Reddit community and earned just shy of 1,000 upvotes. That may seem like a pittance today, but at the time it was enough to get to the top of /r/funny and enough to get onto Reddit’s front page. It was also enough to capture the interest of a literary agent. I remember this last part because sometime after the subscribing to the blog, all of its posts disappeared and rumors swirled of a book that was to come out.

After what felt like an eternity (~17 months), “Where is Bob?” by Irina P. was self published online. Viral sensations have a very short half life, so it wasn’t surprising that the post announcing this publication netted only 10 upvotes. I was able to catch the announcement, and even though it had good reviews, I was too cheap to shell out the $10 or $15 the author was asking for it. I figured my stack of unread books was already high enough.

In the years since “Where is Bob?” was released, my pile has only grown larger, yet one day around 2 years ago I found myself in need of a laugh and thought about the old “Where is Bob?” stories. I decided to look the book up, and to my dismay, found that it and its corresponding blog were gone from the Internet.

I spent several evenings trying to track down the book: Maybe someone had uploaded a PDF, maybe it was still for sale, maybe someone had archived some of the public blog entries… but it was to no avail.

I’m never one to easily back down from a challenge though, so using some clues I found in various corners of the net, I deduced the author’s current contact information and sent her a message. I told her I that I had really enjoyed her stories and I offered, if possible, buy the book directly from her. In response she sent me this message:

Hi Pat,

Yes, indeed, it was I who wrote “Where is Bob.” How cool that you found me, despite my best efforts to stay hidden! “Where is Bob” was a really fun way to express my frustration with certain aspects of IT, which is the field that I used to work in. I even had a literary agent take interest in it, but she had a difficult time selling it to publishers because it was too short to be a novel and didn’t really fit into any other category. So I put it up as an ebook through Lulu Press, because there were a bunch of people who thought the idea was cool and were willing to pay to read the whole thing.

That was a long time ago. I have since switched careers. I currently work as a high school science teacher, and when I began that job, I decided to decrease my Internet presence as much as possible, because I wanted my non-school life to remain hidden and private from my students. I didn’t want them googling and finding that book. It also wasn’t really making any money (not after the initial surge of sales), so I didn’t think there was any interest in it. If I ever write anything again (which is improbable, albeit not impossible), I will publish it as pseudonymously as humanly possible, but if you want, I can add you to the small group of people who will be notified.

Anyway, thanks for the note, it was a pleasure.

All the best,
Irina

There was no mention of selling the book, and I didn’t want to push the issue too much. I understood her point too. I remember being in college and having friends giggle with glee about finding their professor on an online dating site. A saucy book of old IT stories from a previous career might not mix well with science class.

At the time I decided that was that and moved on. A few days ago I again thought of the book and once again I tried finding it, only to come up empty handed. This experience has underscored something that I’ve often thought about that seems to be against conventional wisdom: Stuff does disappear from the internet. As much as we think things will last forever once they hit the web, that’s only the case if the information has a caretaker that wants it around. If it doesn’t have such a caretaker, it’ll eventually either disappear or sink into the deep web.

Personal

Happy New Year!

Leave a comment

Baltimore City Fireworks, New Years Day 2016

I know I’m a day late, but I was tired yesterday. I took this photo just after midnight from the HarborView building in Baltimore, Maryland.

I honestly hadn’t thought much about resolutions until yesterday. I watched a video that recommended making huge goals and then asked viewers the question: “How do you eat an elephant?” The answer was “one bite at a time”. It seems a little silly, but I like that way of thinking about things. Huge tasks are very achievable, you just have to break them down into small pieces and go after them one at a time.

I’m not going to publish my goals here, since science shows that telling people your goals makes you less likely to achieve them, but I’ve decided to set some big goals for myself. I hope whatever goals you’ve set for yourself come to fruition and that you have a great new year!

Personal

The Long Vacation

3 Comments

It’s been a year since I’ve updated this blog. What the hell have I been doing in that time?

Well… my daughter was born, I sold my house, bought a new house, took lots of photos, reentered and won the contest I talked about in my previous post, became much better at photoshop, reviewed a book for Packt Pub, grew a beard, shaved it, and started learning how to use video editing software.

This site has remained on my mind though. In the background of my other activities I think about what I could be doing with it. I’ve been at a stalemate about where to go with it though. When working on it started to feel like work, I drifted away.

In my absence this site has somehow seen it’s best year yet. Traffic is up more than 50% from last year… though truth be told, a big part of that increase in traffic is due to the run away success of my old Snake app. Why that app took off late last year I have no idea. It sat around collecting dust for the longest time, and then slowly started amassing a following. It’s even made an appearance in a CS lab in Ireland. One of my friends suggested that I start making more games, but I don’t want to be chasing traffic.

For the longest time I’ve had this mentality that this site needs to have a focus around “programming”, but that’s not what this site is. It’s a creative outlet. Some of the blogs I wrote in 2012 and 2013 were pretty dry and written with the mentality of this being a “programming blog”. But it’s not. I’m a programmer, and I blog, but this blog is its best when its organic musings about stuff I find interesting and not stuff I think a potential audience might want to read.

So no more book reviews, product reviews, or overly technical posts. This blog will become just a blog. I may write about something new I’ve done, or I may write about my favorite ice cream flavor. And maybe I’ll start to write more, or maybe you’ll see this as the latest entry in 2017. The goal though, will be for this not to feel like work.

With this change in focus I’ve also decided to change the coupling of patorjk.com and patorjk.com/blog. patorjk.com now gets its own page which focuses on the various things on this domain. This will make this blog feel less like a center piece for this site and more like just another section.

So what does the future hold for patorjk.com? Will there be another long vacation before I do another update? I’m not totally sure. The site is always on my mind though.

General News, Personal

Adventures in Photography

4 Comments

It’s been a little quiet around here, but I haven’t forgotten about this place.

Around the start of October I started getting into photography. I had received a Nikon 5300 DSLR camera for Christmas last year, but it spent most of the year collecting dust. However, when the Autumn leaves started falling, I realized it was the perfect time to finally learn how to use the thing.

The Contest

Near the end of October I came across a newspaper article that detailed an amateur photography contest that Kinder Farm Park was holding. The park is run by the county and is 288 acres of forested areas, playgrounds, fields, and farm buildings. Upon finishing the article two thoughts were left running through my mind: I hadn’t realized large scale county parks were a thing, and there were only 3 days left until the competition was over – I needed to find this place and take some pictures.

I ended up making it to the park the day before the competition ended and ran around taking photos like a crazy person. Since it was a Thursday evening, I only had about an hour before it got too dark. I liked two of my photos enough to submit them, and to my delight, two weeks after the contest ended I got an email saying I had scored 2nd place in the People category for this pic:

Before I toot my own horn too much, there were only 9 entries in this category, so it’s not that big of a victory. The other 3 categories had 3 times as many entries, and last year there were a lot more entries, so I’m not sure what happened this year. I guess I got kind of lucky. However, I was still pretty thrilled. I even got interviewed by a local paper, which was kind of cool.

November-December

I’ve spent the past two months basically learning the in’s and out’s of photography (at the time of the contest I was taking all of my photos in auto mode). I resurrected my Flickr page, and was surprised to see that it was still a pretty cool place. Before coming back to Flickr, I briefly tried out Google+, and it was certainly very nice, but Google+ still seems like a ghost town. My belief is that it tried to be too many things (that’s a whole other topic though).

Anyway, I’m certainly not going to change this place into a photography site, but I think changing gears and learning about something new will help me bring more to the table when I finally do come back with some new content. This site is not going to go dormant though. I’ve also spent part of this past year as a technical reviewer for a new Packt Publishing book, so I’ve got an entry coming out about that, and there are a few other small things I’ve been working on, so I’ve got stuff in the pipe line for this site.

Photo Map

And lastly, after the Kinder Farm Park contest ended they released a gallery of all of the submissions. Since each pic listed the location where it was shot, I thought it might be fun to place them into a Google Maps map. If you’re bored you can check them out below.

Site Note: Some of these placements are guesses or approximations.

Visualizations

Rethinking Nutrition Calculators

Leave a comment

I’ve created a Visual Nutrition Calculator that allows you to visually inspect a restaurant’s menu based on the relative calorie, fat, protein, and carbohydrate values of the food.

wwww

The idea is to give a different view of a restaurant’s menu. Rather than picking your meal based on pictures, or choosing a meal and then seeing how many calories it has, the idea is to find and pick items based on their health characteristics. Additionally, a nutrition table is supplied at the bottom that lets you see the stats of your selected choices. You can even view the percent of calories that come from protein, fat and carbohydrates.

Genesis

The bubble chart idea came from a cool presentation by Jim Vallandingham on D3.js’s force layout. I had been looking for another side project to do with the D3 visualization library, and after playing around with Jim’s example bubble chart, I thought it might be fun to try and make my own. I just needed an idea of something to visualize.

Fast forward a few hours and its dinner time and I’m at chipotle.com. I’m toggling the different selections to see how they effect the output and suddenly it hits me! Nutrition calculators have been around on the web for almost two decades. And sadly, they haven’t really evolved much. You check a few boxes, maybe click on a picture of a hamburger, and then you push a button to see how many calories you’ll be eating. The tools don’t really help you make a healthy choice.

Using a bubble chart, I knew I could make a calculator where you could easily see the health impact of each item. It’d be sort of like WiseGeek’s What Does 200 Calories Look Like, except it’d be of what you’re currently thinking of eating.

I started the project shortly before I posted my entry on visualizing bls.gov data and I was originally going to include a whole host of helpful visual aids. My favorite being a sugar cube visualization that would show you the equivalent amount of sugar cubes you’d be eating (similar to Sugar Stack’s How Much Sugar is in Sodas pictures), but I nixed the ideas in favor of just finishing the main idea – I’m bad about starting projects and then not finishing them. Though I still like the sugar cube idea – I think it’d be a cool addition to nutrition calculators.

The Patent…

So I had just finished writing my nutritional calculator and was working on this blog entry when I discovered something that made my heart sink. It turns out there’s a reason that there hasn’t been a lot of movement in the nutrition calculator area, and that’s because back in 2003 the US government granted someone a patent for “Method and system for computerized visual behavior analysis, training, and planning” that covered most of the base aspects of a nutritional calculator. It’s two major claims being:

  1. A system of computerized meal planning, comprising:

    1. A User Interface;
    2. A Database of food objects organizable into meals; and
    3. At least one Picture Menus, which displays on the User Interface meals from the Database that a user can select from to meet customized eating goal.

  2. A system of computerized meal planning, comprising:

    1. A User Interface;
    2. A Database of food objects; and
    3. A Meal Builder, which displays on the User Interface meals from the Database, and wherein a user can change content of said meals and view the resulting meals’ impact on customized eating goals.

    I couldn’t believe my eyes. How could something so general be patented? What was the US patent office thinking?!

    I googled around and discovered the patent holder had been aggressively going after anyone they felt violated their patent. I went back and studied their patent information and sample images. Only their second claim didn’t involve pictures of the food, and that claim seemed to center around a Meal Builder and “meet[ing] customized eating goal[s]”. According to “summary of the invention” section of the patent, the Meal Builder was based around to picture menus. Looking at their sample images (Fig 9), I noticed they had alerts pop up when food parameters like fiber got too high. I reasoned with myself that nothing about mine used pictures or gave advice about goals in general, so I should be safe. But I still felt uneasy. It wasn’t until I came across some recent news stories that I felt comfortable again.

    Bravo Gets the Patent Invalidated

    To my relief, one of the companies sued back in 2012 – Bravo – decided to fight back, and in July of this year they got the US District Court for the Southern District of New York to invalidate the patent. Additionally, earlier this month a batch of Texas lawsuits that the patent holder had filed were dismissed due to collateral estoppel. The patent holder plans to appeal the NY ruling, but for now it looks like its safe again create nutrition calculators.

    Prior Art?

    So near the beginning of this article I mentioned that nutrition calculators had been on the web for almost two decades. Using the Internet Archive, I was able to find one by the Hearst Corporation that existed in 1996. It’s been defunct since early 2007, but finding it took around 10 minutes.

    If these types of apps existed in 1996, how was someone able to patent one in 2003? I’m actually not sure – and in a bizarre twist, the patent holder actually filed a lawsuit against the Hearst Corporation over a meal maker that they created for seventeen.com in 2011. The last I could find on the lawsuit was that it was being transferred to the Southern District of New York along with the Bravo case in late 2013. I’d be curious to know if the Hearst Corporation was aware that they had previously created a nutrition calculator back in 1996. Hell, I wonder if anyone involved in any of the cases actually took the time to see if a nutritional calculator existed in the Internet Archive.

    Another Patent…

    While researching the meal planning patent I also came across another patent that gave me pause. This one involved visual size and nutritional characteristics:

    A system is shown to teach individuals the relationship between the visual size and the nutritional characteristics of portions of food by using either a life size image of, or the corporeal finger of the individual as a scale against life size images of different sized portions of different kinds of food, while showing the nutritional characteristics of such portions; and to adjust the relative sizes of portions of food to provide a nutritionally well-balanced meal.

    This patent is now expired though, so thankfully there was no need to worry.

    Final Thoughts

    Even though small projects are unlikely to be the target of patent holders, it sometimes happens. I’m glad Bravo decided to challenge the meal planning patent. It costs them more money than settling*, but it makes things better for the little guy.

    I think visualization tools like D3.js could be really useful for nutritional sites. This app is really just sort of a proof of concept / prototype. I made it mostly just for fun to see how it would turn out. However, if I were running an actual nutrition site, I could see all sorts of crazy possibilities for something like this.

    * I originally came across the patent when I was using the Jimmy John’s nutrition calculator. The patent information is in small grey text under the “Add to Meal” button.