Field Notes : Syntax Errors


Mismatched or missing delimiters are the most common syntax errors [0xF000]

Syntax error stack traces tell you where the parser failed, not where the mistake was made. [0xF001]

Parsers have no understanding of what your intentions are, they just follow strict grammar rules to transform code into structures and expressions. [0xF002]

Because Ruby is such an expressive language, the rules for parsing it are often more complex than you'd think. [0xF003]

Mismatched or missing delimiters are the most common syntax errors [0xF000]


Source code is just text in a file. It only becomes a runnable program after the parser transforms that text into structures and expressions that Ruby understands.

A syntax error happens when the parser (for whatever reason) cannot complete its job. Among the most simple ways for that to happen is when it can't figure out where one structure or expression ends, and another one begins.

This almost always happens by accident. A single missed quote, bracket, brace, slash, parenthesis, or end keyword can cause the parser to run far beyond the intended end of a structure, stopping only when it happens to match the same symbol somewhere else in the code that doesn't make logical sense, or failing that, at the end of the file.

Related Exercises :: 0x00

Syntax error stack traces tell you where the parser failed, not where the mistake was made. [0xF001]


The challenging thing about this category of error is that because of how the parser actually works, it can only tell you where a particular structure started, and where it gave up trying to parse it.

So the real mistake will end up being somewhere in between those two points. But the error message will almost never tell you exactly where that is.

With that in mind, you can usually find the source of the error by starting at the beginning of the structure and visually scanning the code until you see where the mistake was made.

Related Exercises :: 0x00

Parsers have no understanding of what your intentions are, they just follow strict grammar rules to transform code into structures and expressions. [0xF002]



SPOILER FOR [ 0x00 ] - CLICK TO REVEAL

Note how the error message for problem 0x00 doesn't tell you that the ' at the end of the line is *wrong*, instead what it tells you is that the string literal (which it started to try to parse as soon as it saw the " symbol) was never terminated:

    
    examples/hello.rb:1: syntax error found (SyntaxError)
    > 1 | puts "Hello World'
        |                   ^ unterminated string meets end of file
    
  

This is because it's technically just fine to have a ' inside of a double quoted string,
for example "It's a wonderful world".

And so the parser doesn't try to make sense of what the ' means at all in this code, it just looks for a " to end the string and never finds it, so it hits the end of the file.

SPOILER FOR [ 0x01 ] and [ 0x02 ] - CLICK TO REVEAL
Having a mental model for how the parser will apply a particular rule is helpful.

In the case of single quoted string literals, it's "keep scanning forward in the file until you hit either a backslash or a closing single quote, treating everything else as raw unprocessed text.

In problem [0x01] the intention is clearly to make use of string interpolation.

But in a single quoted string, nothing is treated as a special character except the backslash which is used for escaping purposes (e.g. 'It\'s a wonderful World')

So the string parses fine because it doesn't know or care what's in it, even though it leads to the wrong output by not actually doing the interpolation operation.

This also explains why the parser actually does not immediately fail to complete the string with the missing ' in problem [0x02], but instead, completes it in the wrong place. It happily includes a bunch of lines of code as raw text in the string, only stopping once it hits an unrelated ' farther down the file.

Related Exercises :: 0x00 | 0x01 | 0x02

Because Ruby is such an expressive language, the rules for parsing it are often more complex than you'd think. [0xF003]


You will often have a clear sense of how the code you meant to write works.

The problem with syntax errors is that tiny mistakes can totally throw the parser off in ways that you might never expect unless you have a very deep knowledge of Ruby's more obscure features.

Problem [0x03] is designed to illustrate how that plays out in practice.

It triggers three Ruby features by accident due to the typo:

  • The ability to have an else clause after a rescue clause in a method definition.
  • The ability to define methods within method definitions.
  • The ability to write any Ruby expression directly inside a class definition.

Optional additional details follow inside the spoiler section below.

SPOILER FOR [ 0x03 ] - CLICK TO REVEAL

In isolation, the accidental use of else instead of end in add_item method is easy to spot:

def add_item(item)
  @items.push(item)
else

But if it's invalid code, why does the colorization seem to work as normal? Because in fact, else is a valid keyword to use within a method definition, it just needs to follow a rescue segment.

def coin_flip
  [:heads, :tails].sample == :heads && raise
rescue
  "The coin landed on heads"
else
  "The coin landed on tails"
end

How often is this feature used? To be honest, I can't remember the last time I've seen it in the wild. So I'd guess not often at all.

But it's still a valid language construct, and so it's part of what the parser will look for when trying to break a method definition down into its parts.

Things get even more complicated when you throw into the mix the bit of trivia that method definitions technically can include other method definitions, which do not get applied until they've been executed at runtime:

def coin_flip
  [:heads, :tails].sample == :heads && raise
rescue
  "The coin landed on heads"
else
  def lucky_method_call
    puts "You got lucky!"
  end
  "The coin landed on tails"
end

puts coin_flip

puts lucky_method_call

This code will either print "The coin landed on heads" and then crash because lucky_method_call was never defined, or it'll print "You got lucky!" followed by "The coin landed on tails."

Either way, it would most likely make others reading your code scratch their heads, so it is not recommended.

However, because it is possible to write this program, the parser must allow for it in its ruleset.

Just for good measure, throw in one more Ruby feature that is sometimes not obvious how it works even though it comes up often... the ability to write arbitrary expressions anywhere within a class definition.

class Flipper
  include SomeMixin  #1

  p rand(1..100)     #2
end

When written in macro-like style as shown in #1, it isn't obvious from the outside in that include is actually a method call, not some special keyword.

In fact, it's just an ordinary Ruby expression, so writing something like #2 right in the middle of the class definition is also totally acceptable.

I've used a contrived example here to not get too bogged down in details, but this feature is useful at times -- it's just not one that you'd typically reach for daily, so it might not be front of mind while reading code until something forces you to focus on it unexpectedly.

Taking all three of these Ruby features into account, and changing the indentation of the file to be closer to what the parser "sees" in [0x03], a different shape emerges:

class Bag
  # ...

  def add_item(item)
    # ...
  else
    def take_item
      # ...
    end
  end

  bag = Bag.new
  # ...

Looked at this way, it becomes logical why Ruby crashes with error messages about the else keyword not being valid without a rescue, and the missing end for the Bag class rather than for the add_item method.

The more you understand the range of features Ruby supports, the easier it will be to cut through the noise when you see feedback from the parser that doesn't match your intentions.

But simply being aware of the fact that the scope of possible valid programs is far broader than the scope of reasonable and useful programs will go a long way in breaking your mind's tendency to focus on what it meant to write rather than how code actually works.

Related Exercises :: 0x03


Notes on problems 0x04 - 0x07 will be provided in the paid version of this guidebook.

You can preorder now to be among the first to gain access when it is released.