What can you learn from a side project ⁉️ LZ78, emoji and google cloud.

A 6 minutes story written on Dec 2017 by Adrian B.G.

Meta

One of the most recommended technique to improve your software development skills is the to “do a side project”, but not everyone will tell you exactly what and why you actually can learn from one. I decided to share my experience in building a pet project and its results.

I wrote this article for junior/mid-level developers to demonstrate the benefits of a side-project (pet project).

The trick is to find a proper balance between the known/unknown things you want to build. I mean you can create a side-project with 100% new technologies (for you) but it could be frustrating to make it work and you may quit (EX choosing a different project paradigm, language, IDE, platform etc).

I suggest doing a pet project with a 50/50 ratio of things you master and things you want to learn.

Context 🏝️

️Once upon a time (last week), I had an idea, to modify an encoding algorithm to use emoji. I think this came after I wrote about encoding and held a meetup with my students about encoding in general and protocols.

The next step was to choose the technologies, this was the easy part, I just had to read my to-learn list (it is quite large). I am currently learning Golang (a “cloud” programing language), and I also needed some practice writing unit tests. so this covered the main technologies. The libraries also required a working demo, so the next items from the to-learn list were AppEngine and Jekyll, you can read more details in the next section.

The 3rd step was to seek collaborators, so I involved 1 (out of 2) junior web developers I (try to) mentor, it was a good opportunity to show them some workflow and coding techniques.

The project took around … 30h or so, but the goal wasn’t the end project rather the journey. Another 10h or so were used for the library demo project, which consists of a website with a backend, custom domain, and hosting. The end result can be seen here:

Emoji compress

Unicode is hard ✍🏽

One of the first walls I hit was to deal with Unicode. I haven’t worked with it at a byte level since … forever or never? A nice advantage was that Go literals are UTF-8, so it allowed a more “friendly” code to be written. What have I learned?

more about UTF-8 and Unicode
that characters don’t exist anymore
without the proper libs, dealing with Unicode is painful
that Unicode has a [normalized] (https://blog.golang.org/normalization)form
to appreciate developers that already implement Unicode (like IDE’s)
to appreciate newer programming languages

Compressing 🗜️

Before choosing which compression algorithm to port, I had to do some research. I quickly decided that a dictionary-based algorithm is more suited to this problem and narrowed down the list. After a few other tens of Wikipedia pages, I decided to choose the LZ78 algorithm. Learnings so far:

there are lossy and lossless compressions
many dictionary-based algorithms are used in web development
some methods focus on the speed of the compress, decompress or the size of the archive, each used to solve different problems
smart devs from Facebook, Google and Microsoft created their own algorithms
Snappy is used at Google, Hadoop, MongoDB, Cassandra …
to appreciate simpler algorithms

AppEngine 🚂

Google cloud was on my to-learn list for a while. My only hands-on experiences with it were ~7yrs ago when it launched the PHP support and 1 month ago, at the Google Cloud OnBoard conference. I wanted AppEngine in particular because AWS doesn’t have a similar service (it is somewhere between the EC2 container and Lambda services). What I have learned:

GCloud is … different than AWS in many ways
the Standard environment is very limited
it is fast and easy to work with (1 deploy command __ after the initial setup)
you get many features for free, that in a normal (non-cloud) env you would have to do yourself (rollback, multiple version traffic split, auto-scaling with or without limits, monitoring, tasks, etc …)
I liked the admin UI & docs way more than the AWS ones, they are more user-friendly, it’s a subjective statement ofc
“serverless” is perfect for prototypes, small/medium projects, very small teams, limited scope projects.
it’s mostly free for small projects (even after the 1y trial)
it was a free and good opportunity the check this hosting, as a prospect for a future more larger side-project

I have also come across many horror-user-stories about Google cloud, most of them involving huge invoices, but I think most of the problems can be avoided if you just use the price calculator (their staff uses it too) and do not forget that Google is a profit based business.

Static website 👹

For the demo example I made an interactive website. Jekyll was on my to-learn list since I used Hugo and Github Pages was a no-brainer choice. What have I learned:

Github is awesome, free hosting with a custom domain is a bliss
[liquid] (https://github.com/Shopify/liquid/wiki)is not HTML
that I have to invest more time if I want to deliver a static website as a commercial project_, for example I haven’t managed to make the config variables functional, in HTML or JS._ Most probably I would search a different generator, with some more advanced features.
if webpack or create-react-app has a too big overhead for a project (like a landing page), most probably you will want a Static Website Generator like Hugo or Jekyll.

Vanilla JS6 💀

I started using JavaScript since 2008 or something, but most of the time I have used jQuery and other libraries, mainly because of the lack of browser compatibilities. Now since HTML5, JS6 and most of all Babel exists I can write simple JavaScript, and using only the frameworks that make sense, which in this case was none. What have I learned:

jQuery era is long gone, in a web component project I cannot find its purpose
the new [HTML APIs are quite] (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)interesting
it is good that I have [already switched to JS6] (https://coder.today/switch-to-javascript-6-today-2c8f95e28d60)_(with a different, larger side project)_

Visual Studio Code 🎶

Side projects are a good opportunity to learn a new IDE or new features of the one you are already using. I have been using VSCode for a while now, but with this project I encountered a lot of pesky small issues, only a few of which I remember:

nothing works if your go files cannot compile
Liquid (HTML) support is … bare minimal (no auto format)
I wouldn’t use it for a medium/large project (>50 source code files), it’s simply too unreliable (eg: the autocomplete stopped working many times) compared to a full IDE (I have an IntelliJ license)

Nevertheless, it will remain my favorite multi-language text editor.

Unit testing 💉

Unfortunately, my unit testing skills are lacking, and side-projects are a perfect way to improve any skill. We wrote some tests for the library and the demo web server. Good practice makes it better!

Resources I have used 📚

Thanks! 🤝

Please share the article, subscribe or send me your feedback so I can improve the following posts!

Coder.Today

Coder.Today