Writing a Shell in 25 Lines of Ruby Code

If you use Linux or Mac, every time you open a terminal you are using a shell application.

A shell is an interface that helps you execute commands in your system.

The shell hosts environment variables & has useful features like a command history and auto-completion.

If you are the kind of person that likes to learn how things work under the hood, this post will be perfect for you!

How Does a Shell Work?

To build our own shell application let’s think about what a shell really is:

First, there is a prompt, usually with some extra information like your current user & current directory, then you type a command & when you press enter the results are displayed on your screen.

Yeah, that sounds pretty basic, but doesn’t this remind you of something?

If you are thinking of pry then you are right!

A shell in basically a REPL (Read-Eval-Print-Loop) for your operating system.

Knowing that we can write our first version of your shell:

prompt = "> "

print prompt

while (input = gets.chomp)
  break if input == "exit"

  system(input)
  print prompt
end

This will give us a minimal, but functional shell. We can improve this by using a library that many other REPL-like applications use.

That library is called Readline.

Using The Readline Library

Readline is part of the Ruby Standard Library, so there is nothing to install, you just need to require it.

One of the advantages of using Readline is that it can keep a command history automatically for us.

It can also take care of printing the command prompt & many other things.

Here is v2 of our shell, this time using Readline:

require 'readline'

while input = Readline.readline("> ", true)
  break if input == "exit"

  system(input)
end

This is great, we got rid of the two puts for the prompt & now we have access to some powerful capabilities from Readline. For example, we can use keyboard shortcuts to delete a word (CTRL + W) or even search the history (CTRL + R)!

Let’s add a new command to print the full history:

require 'readline'

while input = Readline.readline("> ", true)
  break                       if input == "exit"
  puts Readline::HISTORY.to_a if input == "hist"

  # Remove blank lines from history
  Readline::HISTORY.pop if input == ""

  system(input)
end

Fun fact: If you try this code in pry you will get pry’s command history! The reason is that pry is also using Readline, and Readline::HISTORY is shared state.

Now you can type hist to get your command history 🙂

Adding Auto-Completion

Thanks to the auto-completion feature of your favorite shell you will be able to save a lot of typing. Readline makes it really easy to integrate this feature into your shell.

Let’s start by auto-completing commands from our history.

Example:

comp = proc { |s| Readline::HISTORY.grep(/^#{Regexp.escape(s)}/) }

Readline.completion_append_character = " "
Readline.completion_proc = comp

## rest of the code goes here ##

With this code you should be able to auto-complete previously typed commands by pressing the <tab> key. Now let’s take this a step further & add directory auto-completion.

Example:

comp = proc do |s|
  directory_list = Dir.glob("#{s}*")

  if directory_list.size > 0
    directory_list
  else
    Readline::HISTORY.grep(/^#{Regexp.escape(s)}/)
  end
end

The completion_proc returns the list of possible candidates, in this case we just need to check if the typed string is part of a directory name by using Dir.glob. Readline will take care of the rest!

Implementing The System Method

Now you should have a working shell, with history & auto-completion, not too bad for 25 lines of code 🙂

But there is something that I want to dig deeper into, so you can get some insights on what is going on behind the scenes of actually executing a command.

This is done by the system method, in C this method just sends your command to /bin/sh, which is a shell application. Let’s see how you can implement what /bin/sh does in Ruby.

Note: This will only work on Linux / Mac 🙂

The system method:

def system(command)
  fork {
    exec(command)
  }
end

What happens here is that fork creates a new copy of the current process, then this process is replaced by the command we want to run via the exec method. This is a very common pattern in Linux programming.

If you don’t fork then the current process is replaced, which means that when the command you are running (ls, cd or anything else) is done then your Ruby program will terminate with it.

You can see that happening here:

def system(command)
  exec(command)
end

system('ls')

# This code will never run!
puts "after system"

Conclusion

In this post you learned that a shell is a REPL-like interface (think irb / pry) for interacting with your system. You also learned how to build your own shell by using the powerful Readline library, which provides many built-in features like history & auto-completion (but you have to define how that works).

And after that you learned about the fork + exec pattern commonly used in Linux programming projects.

If you enjoyed this post could you do me a favor & share it with all your Ruby friends? It will help the blog grow & more people will be able to learn 🙂