Crystal Lang: Macros and how they’re useful

Crystal Language Logo
Logo of Crystal Language

The Crystal Programming Language includes a feature called Macros. Crystal’s Documentation states: “Macros are methods that receive AST nodes at compile-time and produce code that is pasted into a program.”. To simplify this means you can write code that writes more code. This post is a deep-dive into how to write macros and why they’re useful.

What’s an Abstract Syntax Tree (AST)?

To understand how Macros work, you should be familiar with the concept of an Abstract Syntax Tree.

In computer science, an abstract syntax tree (AST), or just syntax tree, is a tree representation of the abstract syntactic structure of source code written in a programming language.

Wikipedia https://en.wikipedia.org/wiki/Abstract_syntax_tree

Before a compiler can do its work it reads through all of the source files. While reading it removes all unnecessary characters, spacing, and such. It adds useful annotations such as a line number in case the compiler encounters an error. This is how stack traces work internally. Normally this process would involve changing the files somehow, whereas we’re changing an object representing the syntax instead. An AST Node is just part of the tree. For example one node might be a simple return statement while another might be variable name that equals three. If you want to learn more I recommend reading over the slides from Prof. Stephen A. Edwards Lecture on Abstract Syntax Trees. You could also try listening to Daniel Sanchezs Lecture on Compilers. Both resources will help you get a better grasp than what I can explain.

A first-look at Macros

Macros are an under-documented feature of Crystal when compared to the other features of the language. They are a powerful feature but are also confusing to use. It is a topic as advanced as meta programming. Before learning how to use macros you should have a good understanding of the other functionality of Crystal first. Macros have some special syntax such as {{name}} to insert code or the value of a variable somewhere into the code defined by the macro and a limited subset of language features (some developers say it’s too limited and needs improvement). Macros are challenging but rewarding to use, if you are still reading this blog post give them a try.

How to create a Macro

When trying out a language feature or if I just want to experiment for a while I type crystal play into terminal and load up the in-browser Crystal REPL. Since Crystal is a compiled language, it waits for the user to stop typing, compiles the code quickly, and returns the results of said code in the browser. It’s not a pure REPL but it serves it’s purpose for me. The Crystal Documentation provides an example macro, consider the following code block:

macro define_method(name, content)
  def {{name.id}}
    {{content}}
  end
end
# This generates:
#
#     def foo
#       1
#     end
define_method foo, 1
foo #=> 1

You use the keyword macro to start a block choose a name and define any parameters, then inside the block use the parameters to define methods. This happens at compile time and not runtime, it takes the data from your source files, expands it using the macro and to oversimplify things it copies and pastes the resulting code into your application. You cannot call define_method at runtime. Remember this when using macros in your programs written using Crystal.

A brief Macro syntax summery

I’ve summarized a few Macro syntax features below to give you ideas on what you can do and how you can do it. Several of these examples are from the official docs on macros (which you should read)

Interpolation

You can use {{…}} to interpolate, an AST node.

Conditionals

macro define_method(name, content)
  def {{name}}
    {% if content == 1 %}
      "one"
    {% elsif content == 2 %}
      "two"
    {% else %}
      {{content}}
    {% end %}
  end
end
define_method foo, 1
define_method bar, 2
define_method baz, 3
foo #=> one
bar #=> two
baz #=> 3

Iterators

macro define_dummy_methods(names)
  {% for name, index in names %}
    def {{name.id}}
      {{index}}
    end
  {% end %}
end
define_dummy_methods [foo, bar, baz]
foo #=> 0
bar #=> 1
baz #=> 2

Use-case: Amber Framework uses Macros to register before and after filters

The Amber Framework supports before_filters and after_filters this allows you to run (or yield) a block of code before and after a request. It accomplishes this using a Domain Specific Language which relies on Macros internally. Take a look at the following code block,

# amber/src/amber/dsl/callbacks.cr 
module Amber::DSL
  module Callbacks
    macro before_action
      def before_filters : Nil
        filters.register :before do
          {{yield}}
        end
      end
    end
    macro after_action
      def after_filters : Nil
        filters.register :after do
          {{yield}}
        end
      end
    end
  end
end

The module is loaded and runs two macros, a before_action macro which defines a method called before_filters with no return value and goes through the filters and yields all of the code blocks, the same occurs with the after_action macro and after_filters method. The copying and generation of code is done at compile-time maximizing performance at run-time. A limitation of macros is that the macro cannot rely on run-time information. The macro generated code can use run-time information.

Conclusion

I hope this post gave you a better understanding of macros. Go give macros a try and see what you can build.

Crystal Lang: Building my first web app with Amber

Crystal Language Logo
Logo of the Crystal Language

Recently I heard about a new programming language called Crystal. Crystal is a self-hosted statically typed compiled programming language with C-like performance with Ruby-like syntax. I quickly fell in love with Crystal and the Amber Framework. This post will talk about my experience using it to develop a web application. This post provides a critical overview of what Crystal and Amber, can and can’t do for you. I don’t want to say Crystal and Amber is perfect. Rather I want to be transparent about the current flaws. I want you to make an informed decision to choose Crystal and Amber.

What is Crystal

Crystal is what Ruby would like if it were statically typed and compiled. Crystal marketed as having C-like performance. In reality using benchmarks its performance is closer to that of Rust and Golang.

One of the appeals of Crystal to me is the compiler detects things like nil-unsafe code. The compiler will throw an error if code isn’t nil-safe.

While the performance benefits of Crystal are impressive it does have risks involved.

  • Crystal only has one full time core developer funded, the rest of the core team works in their free time.
  • Crystal’s funding is from its community and a small group of companies using it.
  • There are few Crystal Shards. Shards are the equivalent of RubyGems & NPM Packages. This can make development of new programs take longer. You’ll have to write more functionality on your own.

Shards are unique in comparison to RubyGems and NPM Packages. They behave similar to C++ libraries. You have to compile it on your own. This has security benefits and it saves the developers of Crystal from having to fund their own package servers. They can let GitHub (GitLab, and similar services included here) handle the load for them.

Does Crystal have a Web Framework?

While Crystal looked appealing to me, its website didn’t mention any web frameworks. A quick Google search found Amber. The Amber Framework uses Crystal’s built-in HTTP Server. Amber adds its own features and CLI tools on top to make things easier. You get all of the performance and type safety benefits of Crystal while using Amber. Amber feels like Rails. Amber is less mature but it fits my needs. It follows the model-view-controller paradigm and includes an ORM called Granite.

Amber has some additional goodies included. For example, type amber database into the command line and you’ll be dropped into a Postgresql shell without having to type out the database name. It’s a huge time saver and many similar things are included.

Using Amber does have its own set of cons to add on top of Crystal’s, these include:

  • As with Crystal, the Amber Framework Developers have little funding (no funding according to their LibraPay Page) and are a small core team (all working on Amber in their free time). Bugs may take longer to patch and there’s less people out there to support you.
  • There is less documentation available in comparison to projects like Ruby on Rails. The docs available online explain most things well.
  • Since there are less maintainers, it takes longer to get a pull request merged into their project. If you want to fix a bug quickly you’re out of luck. You will have to apply your own diffs to Amber after pulling the shard from GitHub.

Onto my first project using Crystal and the Amber Framework

Now that I provided background, I can describe my experience developing an application with it. Previously I had been developing a Rails Application that organizes ROM Hacks of Super Mario 64. I decided as a test of the language and framework I would rewrite the app using it. I was able to port the code over in a matter of hours. Ruby has similar syntax to the language. The rewrite process was copying over controllers and views. I had to modify each controller in view to work in Amber, one by one.

How difficult was the change from ActiveRecord to Granite

Writing the database migration code to copy existing data from Rails to Amber was challenging. I had never written migration code before. After the rewrite was complete page loads were three times faster. And it was a positive and pleasant experience for everyone involved. My users didn’t notice the change other than the new theme and faster page load times.

Was was the end result

You can see the end result of the migration at https://hacks.sm64hacks.com/ and check out the source code at https://gitlab.com/sm64hacks/hackdb. I hope this gives you a nice overview of using Crystal and Amber.

Update

Brian J. Cardiff of the Crystal Language Core Team responded to a post on their community forum. Brian said the following about the amount of paid core contributors and how their donations are used.

“The donations advertised in the homepage goes to Manas. Manas currently doubles that income as stated in https://crystal-lang.org/2017/12/19/this-is-not-a-new-years-resolution.html 1 .
Those allows Manas to allocate time from developers to invest in crystal. Although I might be de most visible Manas developer during the last months, other developers joins in scoped or sporadic efforts as well.
Those activities are not always visible. Sometimes is brainstorming, discussing ideas, preparing material, coding, experimenting, giving feedback, etc. Some of them are: waj, ggiraldez, mverzilli, and matiasgarciaisaia.”

Brian J. Cardiff on Crystal Community Forum