Skip to content

Conversation

@ojameso
Copy link
Contributor

@ojameso ojameso commented Aug 18, 2025

Adds PayloadHandler and makes it configurable so users can build custom payload handling for additional meta data or third party messaging consumption.

Chore is built upon the assumption that messages published to the queue will be handled by Chore. Even though Chore allows for configuration of payload_handler, it assumes a structure of the message that makes it difficult to do processing of messages that's not published by Chore (ie. third party streaming data). For instance, it assumes this job_hash:

{
  "class" => Klass,
  "args" => [arg1, arg2]
}

This makes it difficult to parse if the message does not correspond to this structure, for instance:

{
  "id" => "ID",
  "action" => "update",
  "params" => { "enabled" > true }
}

Chore determines the process to run by constantizing the class in the message which poses problems for messages published without this data.

In order to facilitate the above, the method signature for decoding the message was updated to take the UnitOfWork with quite a bit of data. It also refactors code a bit to place message structure handling into PayloadHandler rather than Job to cleanly separate concerns. This way, a user has control over how messages should be constructed/deconstructed. The code still breaks if you don't follow the original job_hash, but it allows for great flexibility on the decoding of the message without reworking a ton of Chore internals. A custom payload handler may look like this.

module Chore
  class MyPayloadHandler < Chore::PayloadHandler
    # Override Chore's decoding to handle proper message handling for non-chore published messages
    def self.decode(item)
      klass = self.raw_json_queues[item.queue_name]
      if klass.present?
        args = Chore::Encoder::JsonEncoder.decode(item.message)
        # job_hash is defined on Chore::PayloadHandler and returns the expected Chore format for message
        return self.job_hash(klass, args) 
      else
        super
      end
    end

    def self.raw_json_queues
      # Class is stored as string for hot reloading in dev
      @raw_json_classes ||= {
        MyJob.prefixed_queue_name => "MyJob"
      }
    end
  end
end

@ojameso ojameso self-assigned this Aug 18, 2025
@ojameso ojameso force-pushed the npe/add_pubsub_payload_handler branch from 767faea to 9428982 Compare August 18, 2025 22:09
@ojameso ojameso requested review from a team and obrie and removed request for a team August 18, 2025 22:20
@ojameso ojameso marked this pull request as ready for review August 18, 2025 22:20
@ojameso ojameso requested a review from a team as a code owner August 18, 2025 22:20
Copy link
Member

@obrie obrie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before I review further, can you expand in the PR description with some examples of what we're trying to support? There are some interface changes being made here and it's not clear to me why they're needed.

@ojameso ojameso force-pushed the npe/add_pubsub branch 3 times, most recently from 87b2efd to 850df19 Compare September 9, 2025 20:28
@ojameso ojameso force-pushed the npe/add_pubsub_payload_handler branch from 9428982 to a119381 Compare September 10, 2025 18:00
@ojameso ojameso force-pushed the npe/add_pubsub_payload_handler branch from a119381 to f4efdf5 Compare September 10, 2025 20:14
@ojameso ojameso requested a review from obrie September 10, 2025 20:53
@ojameso ojameso merged commit fa22327 into npe/add_pubsub Sep 11, 2025
9 checks passed
@ojameso ojameso deleted the npe/add_pubsub_payload_handler branch September 11, 2025 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants