Along my programming journey, I've found some challenges that I consider very useful to improve my skills. I'll list them here, and update this list as I find more challenges.
Advent of Code is a yearly event that happens in December. It offers 25 programming challenges, one for each day of December until Christmas. The challenges are very diverse, and they are a great way to improve your programming skills.
I really recommend it to anyone who wants to learn a new programming language, as it tackles common programming scenarios, such as parsing, algorithms, and data structures.
CLI tools are a great way to improve your programming skills on a given language. One main advantage I find in CLI tools is that they usually don't require a lot of dependencies, letting you focus on the language itself.
Although it may seem very complex, creating a programming language is a great way to understand how things work under the hood. I highly recommend the book Writing an Interpreter in Go by Thorsten Ball. It guides you step by step through the process of creating your own interpreted language. If you want to go further, you can also buy the second book of the series to learn how to create a compiler.
With the knowledge you gain from creating a programming language, you can also create a markup language, for example, a language to create Markdown or HTML files. I assure you that you will have a lot of fun and learn a lot from this experience.
Additionally, it is also a great idea, once you know how to implement a programming language, to use another language to solve the same problem. This way, you are more focused on learning the new language itself, instead of the problem you are solving.
Another great challenge is to create a terminal text editor. It is a great way to learn about how versatile and powerful terminal applications can be. They don't require any graphical libraries, just the language itself.
I recommend to check out the tutorial on Kilo, a text editor written in C. The tutorial is very well written and very easy to follow.
The tutorial covers the basics of the text editor, but I recommend to go further and add more features to it, such as undo-redo, commands, advanced cursor movement, or copy-paste. We take for granted so many features that text editors have, but when you try to implement them yourself, you realize how complex they are.
Also, I think it is a good idea to revisit the C programming language from time to time, just to remember how lucky we are to have modern languages.
And, as with the programming language, you can also re-implement the text editor in another language.
Video game programming is one of the hardest fields in programming, as they require a lot of knowledge in many areas, such as graphics, physics, and performance. But it is also very rewarding, as you can see the results of your work in a very tangible way.
There are many games that you can implement to learn the basics of game programming, such as input handling, rendering, and the game loop.
The Game of Life is a cellular automaton created by the British mathematician John Horton Conway in 1970. It is a zero-player game, meaning that its evolution is determined by its initial state, requiring no further input. One interacts with the Game of Life by creating an initial configuration and observing how it evolves. This, in addition to being a grid-based game, it makes it a great challenge to begin with.
Also, it is incredible to see how complex patterns can emerge from very simple rules. And, if you didn't know it, it is also a Turing complete machine, meaning that it can simulate any algorithm.
There are lots of tutorials out there on how to implement the game on different languages and graphics libraries.
Another great game to implement is the classic Snake. Its game logic is also very simple, but it offers a greater challenge, as it requires to handle the user's input, which will change the game's state.
The game pace is also very slow, rendering every game state to the screen a couple of times per second. However, you cannot read the user's input at the same time you are rendering the game, as it will make the game unresponsive. This is a great way to learn about multithreading and read the input asynchronously.
I have written a tutorial on how to implement the game in Go using the terminal as the GUI, recycling the knowledge I gained from the text editor challenge.
If you are interested in Machine Learning, there are lots of cool challenges you can implement to learn the basics of the field.
One of the most rewarding challenges I've done is to implement a neural network from scratch, without using any libraries. From creating the network, to training it, and using it to make predictions, it is a great way to understand how neural networks work. It is a very complete challenge, as it requires knowledge in linear algebra, calculus, and programming.
I highly recommend to check out the book Neural Networks from Scratch, which guides you through the process of creating a neural network using Python and NumPy.
Another great challenge is to implement a transformer from scratch. The transformer is a type of neural network architecture that is used in natural language processing tasks, such as translation and summarization.
It requires a fair amount of knowledge in Python and PyTorch. I recommend checking the tutorial on how to build a transformer model from scratch of Anfrej Karpathy. It covers the creation of the transformer, the training, and the process of predicting the next word in a sentence of Shakespeare's work.
You will find out that tokenizing the text in a programming language is a very similar process to tokenizing the text in natural language processing.
Another great challenge is to implement the Snake game, but this time using Reinforcement Learning to train the snake to play by itself. Reinforcement Learning is a very unique field in Machine Learning, as it requires the agent to learn by interacting with the environment, instead of learning from a dataset. It comes with several challenges as how to represent the state of the game, how to reward the agent, and how to train the agent.
It may seem trivial to create the reward function, but you will find that the agent will always find a way to exploit it. Your snake will get stuck in a loop if approaching the food gets it more reward than eating it. And, if you penalize the snake too much, it will deduce that suicide is the best option.
I recommend checking the Gymnasium documentation to learn the basics on how to implement your own environment and train the agent.
Another cool thing to do is to play the game yourself, and then train the agent beforehand with "expertise" data. You will find out that the agent will learn quicker and better.