Speed Up Your Ruby App: Background Processing with Sidekiq

Table of contents

Sidekiq is a background job processing system for Ruby applications. It helps run tasks in the background, making your apps faster and more efficient.
Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails to make background processing dead simple.

See the Getting Started wiki page https://github.com/sidekiq/sidekiq/wiki/Getting-Started and follow the simple setup process. You can watch this YouTube playlist https://www.youtube.com/playlist?list=PLjeHh2LSCFrWGT5uVjUuFKAcrcj5kSai1 to learn all about Sidekiq and see its features in action.

For more information, visit https://github.com/sidekiq/sidekiq

Reasons to choose Sidekiq

• Performance: Fast, uses threads for concurrency (eg procesing 1000 emails in parallel). Sidekiq uses threads, typically faster than ActiveJob's default backends
• Scalability: Easy to scale by adding more workers (eg adding workers to handle increased Black Friday traffic)
• Redis backend: Efficient job storage and retrieval compared to some ActiveJob adapters
• Simple API: Easy to implement and use
• Batches: Ability to group related jobs and track their collective progress
• Monitoring tools: Viewing real-time job stats in web dashboard
• Reliability: Automatic retry for failed jobs (eg automatically retrying failed payment processing jobs), more configurable retry mechanisms than ActiveJob
• Free and open-source: Community edition available (using Sidekiq in a startup with limited budget)
• Richer ecosystem: Larger set of add-ons and plugins available
• Flexibility: Supports various job types and priorities (eg SomeJob.set(queue: 'high_priority').perform_async(args))
• Large-scale adoption: Proven in production environments, the list companies use Sidekiq available at https://stackshare.io/sidekiq
• Rails integration: Works seamlessly with Ruby on Rails

Cases where Sidekiq might not be the best choice

• Memory constraints: Limited RAM on small VPS or shared hosting
• Simple needs: Basic background jobs in a small Rails app
• Database-backed queues preferred: Need for transactional job processing
• Language mismatch: Primary application not in Ruby
• Single-threaded requirement: Application not thread-safe
• Cost considerations: Free alternatives preferred for small projects
• Offline processing: Need to run jobs without network connectivity

Examples of Sidekiq in action

Email sending:

class WelcomeEmailJob
  include Sidekiq::Job

  def perform(user_id)
    user = User.find(user_id)
    UserMailer.welcome_email(user).deliver_now
  end
end

# Enqueue the job
WelcomeEmailJob.perform_async(new_user.id)

Image resizing:

class ResizeImageJob
  include Sidekiq::Job

  def perform(image_id)
    image = Image.find(image_id)
    image.resize(800, 600)
    image.save
  end
end

# Enqueue the job
ResizeImageJob.perform_async(uploaded_image.id)

Data import:

class ImportDataJob
  include Sidekiq::Job

  def perform(file_path)
    CSV.foreach(file_path, headers: true) do |row|
      Product.create(row.to_hash)
    end
  end
end

# Enqueue the job
ImportDataJob.perform_async('products.csv')