How to create a Ruby on Rails gem from your existing code

Introduction

It is almost instinctive for developers to search for solutions that other developers have successfully used when faced with a problem. (Even better if there is a preexisting library that we can plug right into our code and save ourselves some time, right?)

In this blog post, I will take my previous blog post (How to use FilePond with Rails' Active Storage) and turn the code from there into a gem. As you are reading through this post, keep in mind this is my approach. There is not a single way to do this, and you will notice as you read through various source codes of different gems that authors have various preferences. There are, however, some general best practices I will outline.

By the way, if you are new to Ruby land, we call libraries "gems". Rails, for example, is a gem itself. So from here on out, I will only refer to Ruby libraries as gems.

What we are building

If you have not yet read through the previous blog post (How to use FilePond with Rails' Active Storage), I recommend you do so now in a new tab. It will give you some context as to what we are building here.

Briefly, here is a summary:

Create (using the code from our previous post) an integration library to the FilePond JavaScript library. This will consist of both Ruby code for our custom controller and code to load the JavaScript library.
Add tests to check our controller, the loading of the JavaScript code, and system tests to check the functionality of our integration.
Document our gem so that others can use it.
Publish the gem to RubyGems.org.

When to create a gem

Having too many dependencies in projects is a sure way to make things difficult for yourself in the long term. It affects code maintainability and makes upgrading (eg. for a Rails project) harder.

Having said that, sometimes it does make sense to use a gem. Here are a few reasons:

Package code for reuse

If you are adding the same code on multiple projects — say, in a single organization or company — and there are little to no customizations for this particular set of code, then packaging them into a gem may make sense. It aligns this shared codebase that eases upgradability and maintainability.

Typical scenarios might be:

Billing system
UI elements
Deployment boilerplate code

Functionality scope is limited

A scenario you typically want to avoid is creating a gem that adds too much functionality. Gems that are limited in scope are easier to maintain, understand, and use.

Examples of this might be:

API wrapper (eg. external service, database, etc.)
Test helper for mocking network calls
Function specific (eg. currency conversion) utility methods

Like everything else in programming, there are always exceptions to the above reasons and examples. The usual questions I ask myself would be:

Can this gem function on its own, specifically for a small subset of features? Or,
Does this gem add a functionality — although coupled (such as with Ruby on Rails) — that is limited in scope and has a relatively stable implementation?

With that out of the way, let us talk about some general considerations when you create a Ruby gem.

Gem authoring considerations

When you release any code that is meant to be consumed by others, there are several things you need to think about. For our discussion, I assume you are thinking about open-sourcing your gem.

What will I call my gem?

Choosing a name is a personal preference. However, if you are planning on publishing to RubyGems.org, you still want to respect the namespace of organizations and individuals who have an existing trademark.

For myself, I prefer descriptive (ie. boring) names. While consulting with companies, I have seen some very large Gemfile files. Very often, I have seen a variety of gem names where I have had to manually check what their purpose is as a dependency.

For our FilePond integration gem, we will call it filepond-rails.

What license will you choose?

There are many types of open-source licensing models. I recommend you check out the Open Source Initiative's License & Standards page for details. The whole topic deserves its own blog post, so instead of rehashing definitions, let me list a few well-known Ruby gems and their licensing models for reference.

Project	License
Ruby on Rails	MIT
Bundler	MIT
Devise	MIT
Sidekiq	LGPLv3
aws-sdk-core	Apache
Nokogiri	MIT
Faraday	MIT
Puma	BSD 3-Clause

The type of license you choose may depend on your philosophy on open source, a specific business objective, or something else entirely altogether. For the gem we are building in this tutorial, we will use the MIT license.

What Rails version(s) will your gem support?

If you are creating a gem that could be used in any Ruby program, you will need to consider specific Ruby versions as well. However, because the context of what we are building is for Rails, our consideration will primarily focus on the Rails version.

When releasing a gem into the wild, it is expected that you define dependency requirements in the gemspec file (which we will go over in a second). Keep in mind that the more versions of Ruby and Rails you want to support, the more work it may be to ensure backward and forward compatibilities.

If you look at the source code of some gems, you will see code like this:

if Rails::VERSION::MAJOR < 7
  # Implementation for older Rails
else
  # Implementation for Rails 7+
end

For filepond-rails, we will support Rails 7 and above.

Instructions for others to develop or contribute

Another thing to consider is whether you want others to fork and contribute to your project. For open-source projects, this is usually a good idea as it provides a path forward for other developers to offer pull requests and help fix bugs.

To make it easier for others to be involved in your project, I typically like to add Docker set up and clear instructions on how to get the project up and running quickly.

For filepond-rails, this is what I will do:

# To run the gem through the "dummy" app, run:
docker compose up

# To enter the development environment where you can run bin/rails g controller and other commands:
docker compose run app bash

Scaffolding your gem

To create our Rails gem for filepond-rails, there are several ways to go about this.

Probably the easiest way is to follow the instructions at Rails Guide: Getting Started with Engines. There are several starter templates on GitHub you can use as well and is useful for creating generic gems (ie. not Rails-specific).

For our project, we will defer to using the Rails Guide like so:

rails plugin new filepond-rails --mountable

The above rails command comes straight out of https://guides.rubyonrails.org/engines.html#generating-an-engine.

We have specified the --mountable option so that we can "mount" our custom controller (and its routes) to the host application. (The host application is the Rails app that is using our gem.)

The generator creates several files and folders to get us started. Let's take a look at these files:

app directory and an internal structure that looks similar to your regular Rails application (eg. with assets, controllers, helpers, jobs, mailers, etc).
bin directory for holding bin stubs.
config directory with a single route.rb file, which defines the mountable routes for your gem.
lib directory which will typically contain the bulk of all the custom code for your gem. Within this directory currently, Rails has scaffolded a namespaced constant. For our filepond-rails gem, we have a FilePond::Rails namespace. There is also a tasks directory for holding gem-specific Rake tasks.
test directory for our tests.
filepond-rails.gemspec to describe our gem, what it does, the author, licensing, gem server configurations, and dependencies. For more information about this file and the various settings which can go in here, have a look at the Specification Guide at RubyGems.org.
Gemfile specifies any other gems you need for your development environment. Typically, for development dependencies, I use the add_development_dependency directive for the gemspec file.
MIT-LICENSE is the default license created by the Rails generator. Amend or change this as needed to your needs.
Rakefile loads the built-in gem Rake tasks, the dummy app Rake tasks, and any other custom Rake tasks you have defined.
README.md should contain a description of your gem, instructions on using the gem, and other relevant information.

Creating our code

This section will describe the various changes we will make to our scaffolded code and turn those generic placeholder files and folders into a releasable library. Let's get started:

1. Define the gemspec

Our first step should be to customize the gemspec.

You will notice that the version of the gem is defined as such:

spec.version = Filepond::Rails::VERSION

The gemspec dynamically references whatever you define as Filepond::RAILS::VERSION. This is defined at lib/filepond/rails/version.rb:

module Filepond
  module Rails
    VERSION = "0.1.0"
  end
end

For a first release, I usually just leave it as 0.1.0. However, once we have completed the first version of the gem, I will bump this up to 1.0.0 as it represents the first real release.

It is a good idea to use Semantic Versioning (SemVer) here. If you are not familiar with this, I recommend you take a look at https://semver.org/.

Essentially, the "versions" you specify in your software should correspond to some meaning. The following is how it is described on the SemVer website:

MAJOR version when you make incompatible API changes

MINOR version when you add functionality in a backward-compatible manner

PATCH version when you make backward-compatible bug fixes

Note: Rails itself follows a SemVer pattern in how it releases new versions.

For the rest of the gemspecs, there are already good inline comments to help you get started. For our gem, you can have a look at our finished gemspec here: https://github.com/Code-With-Rails/filepond-rails/blob/main/filepond-rails.gemspec.

2. Copying over our code

Because we started with an already functioning codebase for our integration with FilePond, we can copy over our code from there into our scaffolded project directory.

If you are following some sort of test-driven development (TDD) workflow, you may want to start by adding tests. For myself, at least when it comes to Rails gems, it usually comes from pre-existing code that I want to extract from another project, so my workflow usually begins by copying over existing code and then adding tests.

From our pre-existing code (which you can see at https://github.com/code-With-Rails/filepond-demo), we will copy over the following:

ingress_controller.rb which handles specific functionality related to FilePond
routes.rb which defines some routes for our above controller
FilePond references
The JavaScript code to instantiate FilePond

Let's quickly talk about each of these files and how we might modify them to fit in with our gem:

ingress_controller.rb

There is not a whole lot we need to modify here as the existing code already functions. However, we will need to namespace our constant from IngressController to Filepond::Rails::IngressController. This is so that we can keep everything that belongs to our gem in its own namespace and avoid possible naming collisions with the host application.

This file should then be placed at app/controllers/filepond/rails/ingress_controller.rb .

routes.rb

In our original code, we just added our routes as part of the main application. Within a gem, we will need to "mount" it in our host application like so:

# config/routes.rb of our host application
Rails.application.routes.draw do
  mount Filepond::Rails::Engine, at: '/filepond'
  # And other routes, etc.
end

You can refer to the Rails Guide for more information about engine mounting.

Within our own gem's source code, however, we need to configure what routes are mounted. We modify our config/routes.rb like so:

Filepond::Rails::Engine.routes.draw do
  # For details regarding FilePond endpoints, please refer to https://pqina.nl/filepond/docs/api/server

  # https://pqina.nl/filepond/docs/api/server/#fetch
  post 'active_storage/fetch', to: 'ingress#fetch'

  # https://pqina.nl/filepond/docs/api/server/#remove
  delete 'active_storage/remove', to: 'ingress#remove'
end

When comparing this to our original code, you can see that it is essentially the same. We made an improvement here by adding active_storage to identify that these FilePond custom routes are meant to be used with ActiveStorage (which was the intent of our integration). Note that this is a personal (ie. mine) preference.

The key message here is to think clearly about how you name your routes. Because routes are (typically) mounted within a sub-path anyway (recall the at hash option within mount Filepond::Rails::Engine, at: '/filepond' ), things will be properly separated. However, you would still want to be intentionally descriptive in the path's names.

See https://api.rubyonrails.org/classes/ActionDispatch/Routing/Mapper/Base.html#method-i-mount for additional details on the mount method.

Adding FilePond references

This step is specific to our gem, but would apply to any gem you are creating that relies on an upstream (typically JavaScript) dependency.

Ideally, we would be able to minimize including any FilePond source code into our gem and simply tell the end-user to add it themselves. If possible, we can avoid adding any code altogether.

For our specific situation, we will be using importmap which does allow us to reference a specific version of FilePond without adding it to our gem's source. Unfortunately, importmap-rails does not support referencing CSS files.

To make things easier for users of our gem, we will add these CSS files into our assets so they could be included more easily. Specifically, we add filepond.css and filepond.min.css into the app/assets/stylesheets folder.

We also add instructions to our README.md to let users know that they need to do this.

Next, we also need to ensure that the upstream FilePond JavaScript library is loaded. Similar to our original codebase, we add an importmap.rb file to the config directory and modify it like so:

# config/importmap.rb

pin 'filepond', to: 'https://ga.jspm.io/npm:filepond@4.30.4/dist/filepond.js', preload: true

This will "lock" our version of the gem to FilePond v4.30.4. Whenever FilePond is updated, we will need to bump this up to support the new version.

Similar to the CSS assets, we add a note in the README.md to instruct users to add javascript_importmap_tags to their application.html.erb layout file so that the definitions in importmap.rb are loaded on rendering.

Adding our JavaScript code

Finally, we need to add our JavaScript code to link everything together. In our original codebase, we placed all this code here: https://github.com/Code-With-Rails/filepond-demo/blob/main/app/javascript/application.js.

For reference, here it is:

// Configure your import map in config/importmap.rb.
// Read more: https://github.com/rails/importmap-rails
import { DirectUpload } from '@rails/activestorage'
import * as FilePond from 'filepond'

// Get the input element
const input = document.querySelector('.filepond')
const directUploadUrl = input.dataset.directUploadUrl

// Initialize FilePond
FilePond.create(input)

// Set FilePond settings
FilePond.setOptions({
  server: {
    process: (fieldName, file, metadata, load, error, progress, abort, transfer, options) => {
      const uploader = new DirectUpload(file, directUploadUrl, {
        directUploadWillStoreFileWithXHR: (request) => {
          request.upload.addEventListener(
            'progress',
            event => progress(event.lengthComputable, event.loaded, event.total)
          )
        }
      })
      uploader.create((errorResponse, blob) => {
        if (errorResponse) {
          error(`Something went wrong: ${errorResponse}`)
        } else {
          const hiddenField = document.createElement('input')
          hiddenField.setAttribute('type', 'hidden')
          hiddenField.setAttribute('value', blob.signed_id)
          hiddenField.name = input.name
          document.querySelector('form').appendChild(hiddenField)
          load(blob.signed_id)
        }
      })

      return {
        abort: () => abort()
      }
    },
    fetch: {
      url: './filepond/fetch',
      method: 'POST'
    },
    revert: {
      url: './filepond/remove'
    },
    headers: {
      'X-CSRF-Token': document.head.querySelector("[name='csrf-token']").content
    }
  }
})

While the above code may work for our app, it is not appropriate when we are creating a library.

For this, we will support the use of ESM modules. To accomplish this, we modify our JavaScript code as follows:

import { DirectUpload } from '@rails/activestorage'
import * as FilePond from 'filepond'

let FilePondRails = {
  directUploadUrl: null,
  input: null,
  default_options: {
    server: {
      process: (fieldName, file, metadata, load, error, progress, abort, transfer, options) => {
        const uploader = new DirectUpload(file, FilePondRails.directUploadUrl, {
          directUploadWillStoreFileWithXHR: (request) => {
            request.upload.addEventListener(
              'progress',
              event => progress(event.lengthComputable, event.loaded, event.total)
            )
          }
        })
        uploader.create((errorResponse, blob) => {
          if (errorResponse) {
            error(`Something went wrong: ${errorResponse}`)
          } else {
            const hiddenField = document.createElement('input')
            hiddenField.setAttribute('type', 'hidden')
            hiddenField.setAttribute('value', blob.signed_id)
            hiddenField.name = FilePondRails.input.name
            document.querySelector('form').appendChild(hiddenField)
            load(blob.signed_id)
          }
        })

        return {
          abort: () => abort()
        }
      },
      fetch: {
        url: './filepond/active_storage/fetch',
        method: 'POST',
        onload: (response) => {
          return response
        },
        ondata: (response) => {
          return response
        }
      },
      revert: {
        url: './filepond/active_storage/remove'
      },
      headers: {
        'X-CSRF-Token': document.head.querySelector("[name='csrf-token']").content
      }
    }
  },

  // Convenience method to initialize FilePond based on the way this gem expects things to work
  create: function(input) {
    FilePondRails.directUploadUrl = input.dataset.directUploadUrl
    FilePondRails.input = input

    // Initialize FilePond on our element
    return FilePond.create(input, FilePondRails.default_options)
  }
}

export {
  FilePond as FilePond,
  FilePondRails as FilePondRails
}

You can see that the code is essentially the same. However, we export our functions at the end which will allow users of our gem to be able to do this:

// On the host application JavaScript file...
import { FilePondRails, FilePond } from 'filepond-rails'

window.FilePond = FilePond
window.FilePondRails = FilePondRails

const input = document.querySelector('.filepond')
FilePondRails.create(input)

Our goal here is to make it as easy as possible for users to integrate our library. Remember that our target developer audience will likely be Rails developers using vanilla JavaScript or Stimulus.

Note: Recall earlier that I mentioned that as a gem author you need to consider what software dependencies your library will require.

For filepond-rails, I have declared that developers wanting to use this library should be on Rails 7+ and be using importmap-rails. Because importmap-rails relies on the use of ESM modules, this is what filepond-rails will support in terms of JavaScript loading.

Testing using the Dummy app

At this point, you might be wondering: Does this all work?

That is a good question and one which is very easy to answer by bootstrapping a Rails app and following our setup instructions. Within our gem source code, there is a "Dummy" application which we can do just that.

For the finished product, you can take a look at the source here: https://github.com/Code-With-Rails/filepond-rails/tree/main/test/dummy.

Within this app, we create a controller, view, and JavaScript to instantiate our gem's custom JavaScript.

Specific for this, we create:

A simple model for associating ActiveStorage attachments (https://github.com/Code-With-Rails/filepond-rails/blob/main/test/dummy/app/models/some_model.rb)
A HomeController that allows users to update their avatar (https://github.com/Code-With-Rails/filepond-rails/blob/main/test/dummy/app/controllers/home_controller.rb) and associated views (https://github.com/Code-With-Rails/filepond-rails/tree/main/test/dummy/app/views/home)
JavaScript code to instantiate our gem's JavaScript code (https://github.com/Code-With-Rails/filepond-rails/blob/main/test/dummy/app/javascript/application.js)

Once we do all this, we can fire up our server with the following command:

bin/rails server

You should be able to head on over to http://localhost:3000 to see your work.

Note: Check out filepond-rails at https://github.com/Code-With-Rails/filepond-rails if you want to try this out yourself.

Question: Instead of using the Dummy app, how do I test my gem on my host application that I'm developing at the same time?

Answer: To be able to do this, you can reference your gem locally.
# Host application's Gemfile
gem 'filepond-rails', path: '/path/to/filepond-rails-local'
Remember to run bundle install and bundle update if you update the code within the gem's source.

Adding tests

To ensure that our gem works and future updates do not break functionality, we will need to add some tests.

For filepond-rails, we will stick with minitest. You can, however, choose whichever framework (eg. rspec) if you'd like.

Because our application consists of two parts (our IngressController and JavaScript loader code), these will be what we will test.

The controller test is fairly simple and standard Rails controller testing: https://github.com/Code-With-Rails/filepond-rails/blob/main/test/controllers/filepond/rails/ingress_controller_test.rb.

Note: You will notice (if you click on the above test) that I have not mocked out the HTTP request in a specific part of the test: https://github.com/Code-With-Rails/filepond-rails/blob/main/test/controllers/filepond/rails/ingress_controller_test.rb#L11-L13.

Admittedly, this is not an ideal test set up and this could be improved. Having said that, the gem uses GitHub's own CI services and other aspects of the GitHub ecosystem. As a result, I have simply decided to get the tests to fetch a real (static content) URL. Do not do this if you are dealing with expensive API calls, for example.

Next up, we add system tests. System tests, as per Rails Guide:

System tests allow you to test user interactions with your application, running tests in either a real or a headless browser. System tests use Capybara under the hood.

For a user-interface enhancing gem like filepond-rails, we want to ensure that our JavaScript source code loads and functions properly. Because our library

To create a system test, we type:

bin/rails generate system_test uploads

This will generate the corresponding test/system/filepond/rails/uploads_test.rb. This test, which is run by typing bin/rails app:test:system in the command line (as it is not run by default with the rest of the tests), will test things in a headless browser.

Let's modify uploads_test.rb like so:

require 'application_system_test_case'

class UploadsTest < ApplicationSystemTestCase
  setup do
    ActionController::Base.allow_forgery_protection = true
    visit(root_url)
  end

  teardown do
    ActionController::Base.allow_forgery_protection = false
  end

  test 'goes to homepage' do
    assert_selector 'h1', text: 'filepond-rails development'
  end

  test 'FilePond is loaded' do
    assert page.evaluate_script("typeof(FilePond) !== 'undefined'")
  end

  test 'FilePondRails is loaded' do
    assert page.evaluate_script("typeof(FilePondRails) !== 'undefined'")
  end

  test 'file upload via file upload' do
    assert_difference -> { ActiveStorage::Blob.count } do
      page.execute_script <<~JS
        window.filePondInstance.addFile("data:text/plain;base64,SGVsbG8sIFdvcmxkIQ==")
        window.filePondInstance.processFile()
      JS
      sleep(5) # There is a delay in FilePond due to the UI providing feedback
    end
  end

  test 'file upload via fetch' do
    assert_difference -> { ActiveStorage::Blob.count }, 2 do
      page.execute_script <<~JS
        window.filePondInstance.addFile(
          'https://user-images.githubusercontent.com/16937/209497677-a0ace476-a04f-4efb-be8c-6d263ad5d0e0.png'
        )
        window.filePondInstance.processFile()
      JS
      sleep(5) # There is a delay in FilePond due to the UI providing feedback
    end
  end
end

Let us go through some of the less obvious parts of the test code from above.

You will see that we have explicitly enabled forgery protection via ActionController::Base.allow_forgery_protection = true. This is because Rails does not activate forgery protection in the test environment by default. We specify to our setup and teardown blocks to reset it back to false after the tests are done.
In our FilePond is loaded and FilePondRails is loaded tests, we execute JavaScript in the browser to make sure our library is loaded properly. If our JavaScript code has improper syntax (eg. when we modify it in the future), this test should fail.
Next, we test the functionalities of when an image or a URL is added to FilePond. Here, we rely on FilePond's JavaScript API to execute the functionality. Given that all of this is happening in a headless browser, should the tests pass we know that it should also work when a real user is performing these actions.

Publishing our gem

Finally, we have reached the stage when we can publish our gem. Before we do that, let us create a CHANGELOG file. The change log should contain all the major, minor, and breaking changes you have made. I usually date the release as well.

Next, I recommend that you tag version releases just as you would for any other software project.

Example:

git tag v1.0.0

Once you have pushed that up to GitHub, create a release with a good description.

To publish your gem at RubyGems.org where it can be accessed by other users, follow the directions at https://guides.rubygems.org/publishing/.

You will first need to build your gem like so:

gem build filepond-rails

This will create a file such as this: filepond-rails-1.0.0.gem.

Once you have that, you can then do this:

gem publish filepond-rails-1.0.0.gem

Note: You will need an account at RubyGems.org and will need to authenticate your gem CLI so that you can push changes publicly.

Documentation

filepond-rails is a simple and small gem, and so a full-fledged project website is not warranted. For more complicated gems with a large public API, proper documentation should be present.

Tip: Check out https://rubydoc.info/ and add your gem to it there. It is an useful tool to help generate documentation from your code comments and make it publicly accessible to developer users.

Ideas for improvement

As with any project, there is always room for improvement. This blog post only scratches the surface and provides the minimum steps in creating a gem project that you and others can use across multiple Rails applications.

Other things that can be done (in no particular order):

Using the Appraisal to test against multiple versions of Rails
Adding GitHub Action CI (not covered here, but you can check out filepond-rails' repo for how we did it there)
Adding a publish to RubyGems.org release pipeline (so we can skip manually publishing the gem altogether)

Conclusion

Creating an open-source gem for others to use can be a satisfying experience, both technically and professionally.

On a professional level, it is a great way to give back to the community and help others who might be facing the same situation that you had faced.

Technically, library creation also requires you to carefully think about the problem you want to solve and to think through your solution so that it minimizes issues and does not cause issues to end users. For these reasons, I usually like to keep gem libraries small in size with a narrow, surgical focus.

I hope you found this tutorial useful. Again, the source code for the filepond-rails gem can be found at https://github.com/Code-With-Rails/filepond-rails. For the gem itself, check out https://rubygems.org/gems/filepond-rails.