Starting a new Elixir Project
I recently started coding in Elixir professionally. It’s relatively easy to develop and add new features at work where most things are set up and running. However, I wanted to get a better understanding of setting up an Elixir project and tooling from scratch. So I decided to start simple and try to write an Elixir wrapper for a 3rd party API. Here are my thoughts and learnings from starting a new Elixir API Wrapper from scratch.
Bootstrapping the Project
The first thing in any new software project is bootstrapping the project and it’s directory structure. Elixir makes this incredibly easy for libraries using mix. You can get up and running with a library by just calling mix new name_of_project and then mix will scaffold out the entire library and get you up and running with a new mix project including a test framework. You can also use mix to scaffold other Elixir based applications using a supervision tree or to start a Phoenix based web app. For the base mix new documentation you can take a look here
Finding an HTTP Client Library
The first step in my Client wrapper SDK journey was finding a good Elixir HTTP client which wasn’t as straightforward as I would have liked. Elixir being a relatively new and small community is still changing a lot and often packages are deprecated or abandoned. Although I was able to quickly come across several to choose from the three main one’s I came across were HTTPotion, HTTPoison, and Tesla. HTTPotion seemed like an early popular package in Elixir but has since been soft-deprecated and redirected users towards Tesla. HTTPoison I have used a couple of times for making simple HTTP requests and seems to have the largest following but also seemed rather restrictive and opinionated. I have come across quite a few other libraries using HTTPoison but after using it a little I was hoping for something else. The last package, Tesla, is based off of Faraday(a ruby HTTP client gem I have used) which was a big plus to me because I am familiar with Faraday and really enjoy it’s composability. Tesla follows suit and allows for easily changing HTTP library backends, JSON parsers, and even defining your own middleware and using predefined composable middleware. These were all big wins for me and led me to my decision to use Tesla as my HTTP client. More recently I have also ran across Mint which is developed and maintained by some Elixir core contributors which is a promising sign and something I will need to look into more in the future. I chose to build out my own internal Client module and API in my package which would allow me to change out the underlying HTTP client in the future if need be. This is something I would recommend in any language for any HTTP wrapper, that way if need be you can easily switch out the underlying HTTP client with minimal changes to the rest of the application. The full implementation of the client is visible here and the documentation.
Testing and Mocking
I think there are some things Elixir does really well with testing and some things that could be better. Let’s start with the good though. The first great thing is that mix projects come with Elixir’s builtin testing framework called ExUnit that has the bare essentials for testing out of the box. No picking between dozens of open source projects or figuring out the testing framework for your needs. It’s built into all mix projects, maintained by the core Elixir team, and dead simple to get setup and running. For anyone coming from Ruby, it’s fairly similar to RSpec (at least the DSL). And another great benefit of ExUnit is that it comes with async test running. Which can really speed up your testing by running tests in parallel. The only negative to this is making sure that you write your tests to operate in parallel. For the most part you probably won’t encounter any parallel process problems, but for more advanced tests and testing setups this can occur.
Now the not so good, mocking (especially third party API calls). Mocking in Elixir takes a relatively opinionated approach which you can read about from the Elixir creator, José Valim, here. The gist is that you should try and avoid mocking whenever possible, but when it’s not possible then you should create a test/mock module to replace the module you are interacting with(not the one you are testing) so that you can define its behaviour but keep the contract of functionality. Without debating this article or thought here, I think a general statement is that this has led to lack of simple mocking libraries or approaches in Elixir unless you roll your own. Something I generally try to do is mock external API calls for simple, quick, and reproducible tests. In Ruby there’s the VCR project that allows you to record external API calls and replay them for future tests. This allows you to, after the first attempt, not have to rely on the external API and gives other developers the same responses without having to know your credentials. However, a problem with this approach is that it’s hard to catch changes in the API’s functionality/behavior but let’s look past this for this article. Luckily there’s a similar project in Elixir called ExVCR that allows you to record HTTP requests and responses. This was great for stubbing my responses to the external API, however there’s a known issue with this library’s speed. Even after recording the request and replaying them it can take seconds per test, whereas it should be almost instant to load in the recorded response and replay it. This is a known issue with the library and can really slow down your tests even with the async abilities of ExUnit.
Best Practices (Linting, Style, CI/CD)
The next major part of any software project is setting up your CI/CD system and any linters or style formatters. First I’ll discuss the linters and style formatters since this generally should be a part of the CI/CD pipeline. This is an area where I think Elixir really shines. It is incredibly easy to get both a linter and style formatter setup and integrated into your project pretty easily. For one with mix, Elixir comes with a built in formatter. All you need to do is define a formatter.exs(comes with mix new by default) file in the root of your project, and then you can just call mix format and your project will automatically get formatted based on your defined rules. This is a big win for Elixir because it means a majority(if not all) Elixir based projects have the same or a very similar code formats and are therefore easy to follow along and read. No spaghetti code to decipher or maintain. Secondly Credo is an open source static analysis tool for Elixir that is also easy to set up and run to make sure you are following best practices and keeping your code free of code smells and other issues. It just takes two steps of adding the library to your project’s dependencies and running mix credo. You will have to manually fix any issues that come up during a credo run though, whereas the formatter automatically formats your code for you.
Next is CI/CD. Personally I have a lot of experience using CircleCI so this is my goto solution because it’s relatively easy to set up for me and is free for public repos. One negative I encountered here is that the default configs for Elixir projects seem a little outdated or incorrect and I had to make some changes to get things to work. The pluses here though is that with mix format and mix credo I was able to easily put linting checks into my CI/CD process along with my tests. You can see the full CircleCI config here in my repo. Lastly I just want to touch on briefly the ease of adding code coverage to Elixir based projects. The ExCoveralls library makes it relatively straightforward and free to set up for your open source Elixir project.
Documentation
Documentation isn’t an afterthought when writing Elixir it’s a first class citizen with builtin support. This has to be one of the best things for new developers coming to Elixir. You can easily write documentation alongside your code, even using markdown, and it will automatically generate beautiful html files, deploy, and host them for you when publishing a library. This leads to incredibly well documented libraries and a great developer experience for newcomers and veterans. It also allows any developer to easily correct documentation and extend it by creating a PR to update it within the repo. Although Elixir is a relatively small community and new library it provides incredible support for developers through this feature that allows anyone to easily learn and understand what a project or library is doing.
Another huge benefit is that you can even run tests based on examples in documentation. This allows you two kill two birds with one stone. Write great documentation with real examples, and then test those examples directly to make sure they do indeed give you the desired results. All you have to do is add one line to your module’s test file like doctest YourModule and it will automatically run all of your examples to make sure they are valid.
Final Thoughts
In general I would say I think Elixir does a great job when it comes to developer productivity and friendliness. Even with a relatively small community, they have made it incredibly easy to start a project and follow best practices right out the gate so that you are learning the right way to write Elixir. This to me has been an issue with a lot of other languages I’ve used in the past. It’s hard to learn and follow best practices because there are seemingly dozens of ways people do things and no de facto standards in a lot of the tooling. Although I do think in general the tooling and libraries around Elixir may be lacking compared to more established languages, the Elixir core team has put the right fundamentals and building blocks in place for anyone to come in and hit the ground running when learning and writing Elixir. I personally have had a great experience writing Elixir so far and am excited to see the language and community continue to grow!