Category Archives: Introspective

Development Thoughts, Introspective, JavaScript, Personal

Becoming the Maintainer of an Abandoned Open Source Project

May 8, 2021 patorjk 4 Comments

Lots of ink will probably be spilled documenting the stories of what people did during the summer of 2020. With COVID-19 spreading throughout the world and many places going into lock-down, it was an uncertain and strange time. Many people took the opportunity to catch up on old TV shows, learn about investing, or create a side hustle. Me? Well, somehow I found myself becoming the maintainer of one of my favorite open source projects.

Part 1: The Big Rewrite

Almost 4 years ago I was gearing up to rewrite a rather huge AngularJS app. I had been a bit disillusioned with Google’s decision to kill off AngularJS (don’t get me started with Angular 2), and I was looking to build an app that wouldn’t be tied to the whims of Google, since they have a tendency to shut things down. After surveying the landscape, a solution involving the open source libraries React, Redux, and Material UI seemed like a good fit. All that was left was finding a good datatable component…

mui-datatables looked clean and worked smoothly. It made my app look beautiful, and I was excited. To add to this, the official github repo seemed to be buzzing with activity. My first feature request got 12 reactions and a bunch of comments. There was clear enthusiasm about this project, and it stood out amongst all of the Material UI based datatables I evaluated. I felt like I’d found the final piece to my puzzle.

For a while everything went smoothly. I was no rush and my work had me juggling several projects, so the rewrite took place in-between other tasks I was doing. It wasn’t until I was a few months in that I realized I needed an ability that mui-datatables didn’t provide. “No worries” I thought, “I’ll just put in another feature request”, but when I checked the repo, I noticed Greg, the library’s creator, was no where to be seen. Instead, someone named Gabriel was now running the show.

At first this was fine, and I was glad to see someone had taken up the mantle and was moving the library forward, but I soon found myself increasingly frustrated. I was so close to having the library that I needed, yet many of my pull requests (PRs) were left to languish and some updates to the library made my life harder. To add to this, Gabriel wanted to rewrite the whole library, deeming its internals fundamentally flawed. He made a pinned issue announcing this big rewrite (“Version 3.0”), and said it would fix the table’s internals and possibility completely change its API. However, after this announcement he continued to work on features for version 2.x, and in updates months after this announcement he would state that he hadn’t started on version 3 and was still thinking about things.

I was concerned. Once again I looked at other datatables, but I was filled with dread when I concluded that mui-datatables was still the best fit for what I was doing. I either had to stick it out and convince Gabriel to take my PRs, or fork the library and be on my way… and well, I forked it. And that actually turned out to be a pretty freeing experience. But there are downsides to forking that cannot be ignored:

You don’t get updates from the community – you’re on your own.
When you eventually leave your position, you leave the next developer with learning the fork you made. I’ve been in this boat and it’s not a fun boat to be in, especially as the fork gets old and bugs are found.

After the app I had been working on launched, I thought a little more about these 2 points. One of the key reasons for the rewrite (but not the only reason) was to get off of the now archaic AngularJS framework so that future developers could easily jump into the project. But leaving someone with a forked library that was heavily modified seemed to clash with this idea.

I decided to once again check in with mui-datatables to see if there was some way we could reconcile. What I found was 60 PRs, 2 pages of unanswered issues, and someone @-ing Greg and Gabriel, asking if the project was dead. The last release was 3 months old, and it had a number of problems (a malfunctioning resizable column feature, the responsive design was broken, etc etc). After Greg and Gabriel, I was the 3rd biggest contributor to the project, and with all the updates I’d done in my fork, I realized I was probably the most qualified person to take over. To add to this, COVID-19 was spreading around the country and my work had recently closed down. I was at home with nothing to do for the foreseeable future. I needed something to take my mind off things.

I decided to reach out to Greg on Twitter to let him know the situation and ask if I could take over. I’d previously sought him out to ask why he’d left the project so I knew he’d be responsive on Twitter. A week later he handed me the keys. I then went to 2 of my oldest PRs, merged them in, and did a release. I was the new maintainer.

Part 2: mui-datatables 3.0

Gabriel briefly returned to wish me well. He told me his time had become scarce, and he simply didn’t have enough of it to spend on the project. Even though I’d felt a lot of frustration with some of his decisions, we’d been friendly and I appreciated the work he’d done. Had he not taken on the project, it most likely would have died. However, a year had passed since he’d made his announcement about the next version of the library. Was a version 3 still on the table? I’d done a lot of work on my fork and there were a lot of PRs. Maybe there was enough new stuff to justify a big release.

For the next two weeks I poured through the PRs and open issues. It was oddly fun and proved to be an eye opening experience. Some of the stuff people submit is completely nonsensical while other things are highly complex and clearly had a huge amount of work put into them. For example, one person had rewritten the whole library in TypeScript, and while this was a neat idea, it was completely impractical and would have made it almost impossible to merge the other updates in (plus, I’m not completely sold on TypeScript – but that’s another story). On the opposite end, there were some requests that were small with little explanation. Often times they didn’t work at all and/or caused the tests to fail. It was like people submitted their work without checking it.

I tried to be nice. After all, each PR represented someone spending their own free time to better this library. At the very least they deserved someone trying out and reviewing the changes that they made – even if they had to wait a year. And to my delight, people were pretty cool. I either got a positive response thanking me for looking at their PR, or no response at all.

Of the 60, I ended up accepting 23. The vast majority of these were bug fixes and minor updates. The only submission that really fell into the “cool new feature” category was one that added an injectable component feature. During this period I also ported over most of the updates from my fork, which in the end, accounted for all but 5 of the new features/API updates. A thorough review of the code base was also done which cleaned up a hand full of anomalies. For example, most of the deprecation warnings had accidentally been disabled in version 2.14.0, and starting in version 2.13.1, a large 5MB file was accidentally being included in the npm package. No one seemed to have noticed these things though.

I also updated the library from using version 3 of Material UI to version 4. In the year that had passed, most of the Material UI community had moved on to version 4. Not updating the library to correctly work with version 4 had probably hurt adoption. When I had parted ways the previous year, mui-datatables had 15k downloads per week on npm and it’s closest competitor, the feature heavy material-table, had 7k a week. Now the tables had turned. Material-table was crushing it at 80k downloads a week to mui-datatable’s 25k. The version issue most definitely wasn’t the only reason mui-datatable had lost ground, but I have to imagine it was a significant factor in people’s decisions.

In the end, version 3 would be no great revelation, but it would be a step forward and hopefully a step in the right direction.

Part 3: The Rise and Fall of a Maintainer

After it was released a few people chimed in to say thanks and report bugs, but there was no big celebration that the library was back to getting updates. I got the impression that many people using the table had built their apps around it a year ago. It didn’t seem like it was attracting a ton of new users. From github stars, I could tell that on average, 1.35 new people were starring the library a day, which seemed a bit lower than it had been in the past.

To get things going again I decided to start work on two features I felt were essential:

A cell rendering method that would significantly boost performance.
Draggable columns.

I had a soft spot for draggable columns. It had been discussed with feverish enthusiasm when I was originally looking at the table, and I felt like it would be poetic as the first big feature of the 3.x era. I wanted something that was nice though, I didn’t want something cheap looking. So I got to work and created what I felt was a slick draggable column feature:

click to play or pause

Most people will say they don’t care about small little effects like this – that all they really want is the functionality, but over time I’ve found this to be false. Little touches like this add up, and overall lead to users liking a product more.

As I prepped a new release, I began to talk about new features with the few of the people still hanging around. Maybe a grouping feature should be next? Editable columns? I was excited. I was going to get this table back on track and soon it would have features that rivaled material-table. But in addition to the lack of activity in the repo, something else bothered me: Surely I couldn’t be the only one who felt datatables for Material UI were lacking? Hell, when I was doing AngularJS there were several great community options. What was going on?

I went digging, and found my answer on the material-ui github repo. In a thread lambasting material-table, several people stated that they weren’t happy with the community options. The co-creator of Material UI, Olivier Tassinari, satiated the criticism by ensuring them that they were hard at work on an official datatable component. It would be ready for preview in September – basically at the end of the summer.

I had been out of the loop, and though it seemed obvious, apparently the community was displeased with both mui-datatables and material-table. The creators of Material UI realized they needed an official solution, so in October of 2019 they’d announced plans to create one. That explained why no other table had come forth, and it made me feel like mui-datatables and material-table were both lame ducks. A good solution was on the way, and there was no point in a community project if an official table was going to be supported. (however, I would later learn that certain parts of this official table were for paid users only – so material-table and mui-datatables would still have a place in the future)

I was a little distraught, but decided to continue work, albeit at a slower pace. Then, after 10 weeks of being at home, I got the call to come back to work. I reintegrated mui-datatables into work my project and showed off some of the new features. My team lead seemed impressed and was thankful we were no longer using a forked project. With work on mui-datatables now restricted to evenings and weekends, my contributions to it slowed even more. Then, one day in late September, I handed in my resignation at work.

Wait, what?? Oh yeah, probably forgot to mention that during my time off I was kind of stressed about my future. My work had been extremely generous in paying us to stay home and do nothing, but there were rumors about leave without pay in the Fall. With no telework option available and no assurances made about what might happen once the leaves started turning, I had decided it was best to hedge my bets and look for another position.

In a twist of fate my new job would involve working on another AngularJS 1.x project (it never ends!) and possibly porting it to React (though as of now that hasn’t happened – my guess is we’ll still be dealing with AngularJS 1.x apps 10 years from now, though that’s a topic for another time). Now I had even less motivation to continue work on mui-datatables. I didn’t want the project to fall back into disarray, so I felt like the only reasonable thing to do was to find a successor. Thankfully during my time as maintainer another developer, Woo Dohyeong, had joined me in my quest to better the library. He was the obvious choice to take over and with Greg’s blessing, I passed the torch. After Woo made his first release I stepped back.

It was bittersweet. Part of me knew I didn’t have enough time to be a maintainer forever. It’s a job that gobbles up the extra minutes of your day and its mostly thankless. I didn’t talk too much about it above, but there were a few days where I would handle half a dozen questions, and the majority of people wouldn’t say thanks or respond, some people would even be rude. However, reviving the table and improving upon what so many others had built was rewarding. There was a sadness in stepping back, but it was for the best.

Final Thoughts

Well, I didn’t expect this to be so long, but the story (even trimmed down) turned out to be much longer than I thought it would be. I needed a place to write it down though, and if you read it, thank you for reading my story. The summer of 2020 was a crazy time, and even this bloated blog post barely scratches its surface. Hell, I didn’t even write about my bike rides through empty streets or obsession with Hollow Knight (and Animal Crossing, and Cuphead), not that those things are in any way relevant to mui-datatables, but they filled the gaps between development. Anyway, hopefully you found this entertaining or enlightening. Next time you use a piece of open source, be sure to show appreciation to the maintainer, and don’t be afraid to contribute yourself. If you have the time it can be a fun little adventure. Also, don’t get too caught up in the endless cycle of front-end rewrites. There’s a certain madness to it.

Introspective, Personal

Where is Bob?

January 28, 2016 patorjk Leave a comment

You know you’ve reached a strange point in your life when you find yourself reminiscing about stuff you saw on Reddit almost a decade ago. However, sometimes something you read or see online just sticks with you. It resonates. Whether it makes you think, feel, or laugh, it leaves an impression. The “Where is Bob?” blog, which appeared on Reddit back in 2008, was such a thing for me.

Penned by a wickedly funny IT worker, the Where is Bob blog hosted a set of possibly non-fiction stories about a work place run by an obnoxious and often absentee manager named Bob. The submission garnered the praise of the Reddit community and earned just shy of 1,000 upvotes. That may seem like a pittance today, but at the time it was enough to get to the top of /r/funny and enough to get onto Reddit’s front page. It was also enough to capture the interest of a literary agent. I remember this last part because sometime after the subscribing to the blog, all of its posts disappeared and rumors swirled of a book that was to come out.

After what felt like an eternity (~17 months), “Where is Bob?” by Irina P. was self published online. Viral sensations have a very short half life, so it wasn’t surprising that the post announcing this publication netted only 10 upvotes. I was able to catch the announcement, and even though it had good reviews, I was too cheap to shell out the $10 or $15 the author was asking for it. I figured my stack of unread books was already high enough.

In the years since “Where is Bob?” was released, my pile has only grown larger, yet one day around 2 years ago I found myself in need of a laugh and thought about the old “Where is Bob?” stories. I decided to look the book up, and to my dismay, found that it and its corresponding blog were gone from the Internet.

I spent several evenings trying to track down the book: Maybe someone had uploaded a PDF, maybe it was still for sale, maybe someone had archived some of the public blog entries… but it was to no avail.

I’m never one to easily back down from a challenge though, so using some clues I found in various corners of the net, I deduced the author’s current contact information and sent her a message. I told her I that I had really enjoyed her stories and I offered, if possible, buy the book directly from her. In response she sent me this message:

Hi Pat,

Yes, indeed, it was I who wrote “Where is Bob.” How cool that you found me, despite my best efforts to stay hidden! “Where is Bob” was a really fun way to express my frustration with certain aspects of IT, which is the field that I used to work in. I even had a literary agent take interest in it, but she had a difficult time selling it to publishers because it was too short to be a novel and didn’t really fit into any other category. So I put it up as an ebook through Lulu Press, because there were a bunch of people who thought the idea was cool and were willing to pay to read the whole thing.

That was a long time ago. I have since switched careers. I currently work as a high school science teacher, and when I began that job, I decided to decrease my Internet presence as much as possible, because I wanted my non-school life to remain hidden and private from my students. I didn’t want them googling and finding that book. It also wasn’t really making any money (not after the initial surge of sales), so I didn’t think there was any interest in it. If I ever write anything again (which is improbable, albeit not impossible), I will publish it as pseudonymously as humanly possible, but if you want, I can add you to the small group of people who will be notified.

Anyway, thanks for the note, it was a pleasure.

All the best,
Irina

There was no mention of selling the book, and I didn’t want to push the issue too much. I understood her point too. I remember being in college and having friends giggle with glee about finding their professor on an online dating site. A saucy book of old IT stories from a previous career might not mix well with science class.

At the time I decided that was that and moved on. A few days ago I again thought of the book and once again I tried finding it, only to come up empty handed. This experience has underscored something that I’ve often thought about that seems to be against conventional wisdom: Stuff does disappear from the internet. As much as we think things will last forever once they hit the web, that’s only the case if the information has a caretaker that wants it around. If it doesn’t have such a caretaker, it’ll eventually either disappear or sink into the deep web.

Introspective, Personal

Merry Christmas and Happy Holidays!

December 24, 2009 patorjk

I’ve been bad about updating the past couple of months, I should have some interesting stories for next year though.

It’s been a pretty amazing year. I got engaged, bought a house, wrote a couple of new web apps, learned a lot about Tcl and JavaScript, visited Northern California (and then got stranded in Phoenix on my way back – due to the blizzard going on in Maryland), worked over 180 hours of overtime, and gained a lot of weight (the last one’s not a good thing, but it’ll give me a goal of losing weight for the new year).

I looked up my resolutions for this past year and I think I did ok. Here’s the list:

Get good at JavaScript: I feel like I’ve accomplished this one. There’s more for me to know, but I feel like I really know my way around the language.
Learn jQuery: Didn’t even look at it this year. There are a lot of cool JavaScript frameworks out there though. I’ll probably get around to learning jQuery at some point, but I don’t think knowing it is as important as I used to.
Develop More Web Apps: The Typing Speed Test, the HTML Canvas game, the Blog Stylometry Tool, and lots of updates to the Keyboard Layout Analyzer. I was sort of quiet near the end of the year, but I expect for things to pick up again here soon.
Read At Least 4 Programming Books: Learning Dojo, Learning ExtJS, JavaScript: The Good Parts, and Code: The Hidden Language of Computer Hardware and Software. JavaScript: The Good Parts and Learning ExtJS were the best of that group, and after thinking more about it, probably the only two I’d recommend from the list.
Join A Programming Community: I did not do this, unless you count my subscription to the Programming Sub-Reddit at reddit.com. I’ll give myself half credit here.
Write A WordPress Plug-In: I had a couple of ideas, but I lost interest.
Release Some Programming Examples: I still want to do this.

So I achieved 3.5 of my 7. Not too shabby, I feel like I accomplished what was important to me, and that’s probably better than meeting the goals I think up at the beginning of the year.

This probably wont be my last post of the year (I’ve got a book review coming), but I hope everyone out there has a great holiday and a happy new year!

Introspective, Personal

Two Years Since The Relaunch…

March 21, 2009 patorjk 4 Comments

As of March 21st, it’s officially been two years since I decided to relaunch this website. On the opening day there wasn’t any hype and the site had been down for 2 months – and before that, I think I’d only updated twice in the previous two years. Plus, on this new day, all I had up was a message saying I’d be back. I couldn’t post up anything because my previous host had deleted all of my content and my hard drive had just recently crashed, taking with it everything I had saved. Other than being a lesson in routinely backing up files, this gave me a chance to start fresh.

This site, shortly before being deleted

It’s been two years and I think this site is currently as successful as its ever been. I don’t have a community like I used to, but I do have a decent amount of traffic and links coming in. The site, as a whole, currently gets around 2,700 visitors a day. So I feel good that I’ve been able to come back with something that people (hopefully) find useful and/or entertaining. I suppose it’s just a different site now than it used to be. I’m still steering the ship, and I’m still programming, but I’m using different tools and doing different things.

I think the thing I’ve enjoyed the most over the past two years is seeing the stuff I make or post up actually being used. When working in a big company, sometimes you program something up, hand it in, and then move onto your next assignment. You don’t really get to find out the experiences of the end user or what they thought of the program. You don’t know if they struggled to understand what you did or if they loved it. Well, sometimes you find out, but not always.

The cool thing about the web is that anyone can see the stuff you make. And to get feed back one can just google around or look at their referral logs and see where people are coming from. Sometimes this leads to some interesting info about how people are preciviing and using what you have up. Check out these Stumbleupon reviews I found of my 179 Ways to Annoy People chain letter page:

Annoyed Visitors

I think a certain number of visitors were thinking it was a real list. I thought it was obviously a joke, but you never really know what people are thinking when they come across your site, so I added in note at the top of the page letting people know it was just a joke list. Though I suppose some of the reviewers might have realized it was a joke and were just plain offended by it, however, calling a joke list “Asshole Training” seems a bit odd. [As a side note: I’m thinking of getting rid of this page entirely, since it doesn’t really go with the rest of the site, though maybe I just wont link to it and let its only source of traffic be 3rd party services like search engines and Stumbleupon]

Another interesting observation I was able to make, this time via my stats, was that a decent number of people were going to my Text Ascii Art Generator program and then just exiting via one of the out going links on the About page. This seemed like a rather strange behavior so I changed the program to automatically display the message “Type Something” once the page was loaded. This ended up causing a big increase in the usage of the program. My guess is that people were visiting the app’s main page and then getting confused as to what they were looking at, so they’d just skim the page and click on one of the out going links. It’s stuff like this, as a developer, that I’d never catch unless I had access to usage stats or user feed back.

Anyway, for those of you who’ve decided to follow this site for whatever reason, I thank you for checking in every so often. It definitely is nice to see stuff I make being used and to get feed back on it.