Ticket by Ticket

In the first version, moving one support ticket from Zendesk^® to Jira^® took about seventy-nine seconds.

The number makes more sense once you look at what a ticket involved. Before an issue could exist in Jira, the accounts the ticket depended on had to exist there too, so the tooling handled or created those first. Then it created the Jira issue. Then it migrated the ticket's comments, maintaining the relationships between them. Only when all of that had finished did the next ticket begin.

So a ticket was a small workflow with an order to it, not a record to copy. Across more than ten thousand of them, that had two consequences. At the original sequential rate, simple arithmetic said that processing them all would take days, not hours. And anything that runs for days has to be able to stop and start again without losing track of what it has already done.

The tooling worked with both platforms through their REST APIs: Zendesk's Ticketing API on one side and Jira's REST API on the other. The migration covered dozens of organisations, and it was real operational data, which is why reliability mattered as much as it did.

The first version ran one ticket at a time and had no mature way to pick up where it left off. The later tooling tracked every ticket, recovered from being stopped part-way, and could run up to six ticket migrations at once. Most of this piece is about how it got from one to the other, and the trade-offs along the way.

A note on words: throughout, a ticket is the unit being migrated, and an issue is the record it became in Jira.

The source and target models did not line up one-to-one, so moving a ticket meant deciding what it should become in Jira. Custom fields and schema differences needed explicit mapping decisions, and they took a disproportionate share of the effort. Together, those decisions defined what the migrated data would look like to the people using it in Jira.

Identity took a similar share. A ticket refers to people, and a Jira issue can only refer to accounts that exist in Jira, so account handling came first in each ticket's workflow and had to be right before the rest could follow.

I didn't work the mappings out alone, and they weren't purely technical. I worked with support, product, QA and operations colleagues to understand the source and target workflows, the schema differences and the operational edge cases. The mappings had to represent how those teams actually used the two systems. Workflows are the clearest case, because a workflow describes how a team works rather than what a record contains.

The first version was sequential and comparatively simple: take one ticket through the workflow, finish it, then move to the next. It had basic logging and some retry handling and validation. With one ticket in flight at a time, any problem belonged to exactly one ticket.

It also had a limitation. Each ticket involved several separate operations against Jira: accounts, then the issue, then the comments. If a later step failed, the earlier ones had still happened, and an early version could leave a partially migrated issue behind, with some of its comments and not others.

As the migration became more operationally significant, the tooling matured together across reliability, observability, recovery and performance. Resume state and more robust recovery belonged to the later tooling, while the move to six concurrent ticket workflows was a major performance step.

The arithmetic from the opening made recovery important. A job measured in days has to expect interruption, and each interruption raises the same question: what has already been done? The next two sections deal with that question. Concurrency comes after them, although in practice these developed alongside each other.

I designed and implemented migration state and resumable execution as the tooling matured.

Progress was tracked per ticket, the same unit the rest of the workflow used, rather than per organisation or per step. It was persisted locally so that it survived the process stopping. Restarting the tooling no longer meant restarting the migration, and long runs did not have to be treated as disposable one-shot scripts.

On a restart, the tooling had two records to go on. Its own state described what it had recorded as done, and a process can only record what it knows it has done: if it stops between doing something and writing it down, its record falls behind. Jira showed what actually existed, but an issue being there did not, on its own, say whether the migration had finished with that ticket.

The tooling used both. It combined its per-ticket state with checks against Jira to decide what still needed processing.

The per-ticket logs sat alongside this. When someone needed to understand what had happened to a ticket, they read the logs; when the tooling started again, it read the state.

This was practical recovery logic: local state, a check against the destination, and a decision per ticket about whether it still had work to do.

A restart could also find a ticket that had only half arrived, with the issue in Jira but not all of its comments. This was the same state an earlier version could leave behind.

Recovery worked at ticket level. If an issue existed but its comment migration was incomplete, the ticket was reprocessed with its complete comment set, rather than resumed from a particular comment.

Looking back at that design, the trade-off is clear. Working at ticket level meant the tooling only had to keep state per ticket, not per comment, and that kept the recovery behaviour relatively easy to reason about. The cost was that the replay covered the full comment set, so comments that had already reached the issue could be written a second time. Those duplicates had to be cleaned up afterwards.

Avoiding the duplicates at comment level would have meant keeping a record for every comment, and more recovery logic to maintain it. That is a cost too, just a different one.

The partially migrated issue that an early version could leave behind had become something the later tooling had a defined way to finish.

Back to the seventy-nine seconds. In the first version, only one ticket workflow was ever active. Whatever a ticket spent its time on, nothing else moved while it did.

I implemented controlled rolling concurrency so that up to six ticket migrations could run simultaneously. When one ticket finished, the next began, so up to six workflows were in flight at once. Six was a ceiling rather than a target: the tooling never had more than six ticket workflows open at the same time.

The unit of concurrency was the ticket, and the order inside each ticket stayed the same: accounts first, then the issue, then the comments, because each step depended on the one before it. What changed was that several tickets could be at different stages at the same moment. One might be handling its accounts while another was creating its issue and others were migrating comments. In hindsight the ticket was a natural boundary for this, since each one already carried its own ordering.

With several ticket workflows interleaving, logs stayed tied to individual tickets. A single chronological stream was no longer enough to understand which operation belonged to which ticket's migration.

After the change, the effective processing time per ticket was about 14.7 seconds, against about 79 seconds for one complete workflow in the sequential version. That is roughly 5.4 times the throughput, measured on reasonably comparable workloads.

The 14.7 seconds needs reading carefully. It is the total elapsed processing time divided by the number of tickets completed. An individual ticket still had its accounts, its issue and its comments to get through, and could take longer from start to finish. What the concurrency changed was throughput, the number of tickets the tooling completed in a given time. It did not make any single ticket take 14.7 seconds.

Concurrency was a major part of the improvement, not necessarily all of it; the implementation as a whole was also improved over time.

Across the migration workload, the tooling achieved approximately 99.8% migration success. Failures were logged rather than stopping the run.

A second version would start from a few different assumptions. The first two are things the later tooling already did; the change would be doing them from the start.

Migration state would be a first-class concern from the first version, not something the tooling grew into. A migration of this size is a long-running process from the beginning; it just doesn't look like one at small scale.

Logs would be structured around ticket identity from the start. Concurrency made that necessary, but it is useful long before there are six tickets in flight.

I would look at stronger protection against replaying comments that already exist, so that recovering a whole ticket keeps its simplicity without leaving duplicates to clean up afterwards. How far to take that is a judgement call, because per-comment tracking has costs of its own.

And I would think explicitly about shared dependencies, such as accounts that several tickets refer to, before running tickets side by side.

The decisions that mattered most in this migration were about what a ticket should become in Jira, which accounts it depended on, what the tooling should remember about it, how to recover it when it had only half arrived, and how many ticket workflows could be in flight at once. The API calls carried those decisions out.

The mappings came out of work with the teams who would use the result. As the migration became more operationally significant, the tooling also gained per-ticket state and resumable execution.

The unit that mattered was the ticket, not the API call. It was the unit of work, of state, of concurrency, of logging and of recovery.