Guide · Operations
One studio failing is not a site outage
Studios have incidents of their own. The catalogue should mark that studio unavailable and leave every other studio launchable. A lobby that dies because one feed died was built as a single point of failure.
7 min readUpdated By the Turnkeyxgaming team
Hide, don't proxy the error into every tile
When launches for a studio start failing, remove that studio's tiles or show them as unavailable. Do not let the client spin on a timeout. Players with an open round need a different message from players browsing: their round will void or settle when the studio recovers, and the wallet rule for that case is the rollback path. Inventing a result because the studio is quiet will not match the eventual settlement.
Status communication belongs on your side. A banner that names the studio is more honest than a site-wide maintenance page. Keep a note of the start time. When the studio returns, enable the tiles again. Do not leave yesterday's incident flag on a healthy studio, and do not quietly turn on a substitute game under the same tile.
Decide the flag from launch failures for that studio, not from a single slow spin and not from social chatter. A threshold you wrote down, over a few minutes, beats a manual panic that also hides studios which are fine. Players who are only browsing get the unavailable tile. Players with an open round get a different sentence: the round will void or settle when the studio comes back, and the balance will follow the rollback or the credit. Those are different conversations.
Who needs which message while one studio is down
The lobby stays up. The copy changes with whether this player has money in an unfinished round at the studio that failed.
| Who | What they see | What you must not do |
|---|---|---|
| A player browsing | That studio's tiles unavailable, other studios normal, a banner that names the studio. | A site-wide maintenance page, or a substitute game under the same tile. |
| A player with an open round there | The round will void or settle when the studio recovers. The wallet follows that outcome. | A made-up result so the spinner stops, or a goodwill credit before the status exists. |
| A player in a different studio | Nothing about this incident, unless their game is actually affected. | A global error state inherited from one failed feed. |
| Support, after recovery | Start time, end time, and the flag cleared on purpose. | Yesterday's unavailable flag left on a studio that is launching again. |
An incident that stays small
- 01
Studio flag
Mark the studio unavailable. Leave other studios alone.
- 02
Open rounds
Tell affected players the round will void or settle. Do not invent a result.
- 03
Banner
Name the studio on the banner, and take the banner down when launches work again.
Do not invent a settlement because the feed is quiet
Silence is not a result. A credit or a rollback will arrive when the studio recovers, and it will not match a balance you already 'fixed' to calm the player. Tell them the round is unfinished.
Six minutes of failed launches and thirty-seven open rounds
At 21:40 launches for one live studio began failing for about one player in eight. The on-call read the first two as a player's network, which is reasonable, and had a pattern by 21:52. The studio flag went on at 22:04. In between, the lobby kept offering tiles that half-worked, and 37 rounds were left open at that studio holding stakes the wallet had already debited.
The flag is the easy part; the queue of open rounds is what the incident is about. At 22:30 an agent, with a player on the phone and a team lead's blessing, credited four of those rounds at 46 each, because the player was upset and the studio was silent. At 23:05 the studio returned and sent credits and rollbacks for the same rounds. Two of the four arrived as rollbacks of debits the agent had already replaced. That player ended the evening 92 up on rounds that never completed, and finance found it a fortnight later.
Write the rule before the incident: no manual credit against a round id that appears on an incident list. Generate the list instead of typing it. Export the open rounds when the flag goes on, put the studio id and the round ids where support can see them, and give one person the authority to close them as the callbacks arrive. A goodwill payment, where the brand offers one, is a separate labelled credit after the round is closed, with the round id in the note so nobody pays the same gesture twice.
The sheet opened when the flag goes on
Completed at the start of the incident, not reconstructed the next morning from chat.
- The studio id and the catalogue name players see, so the banner and the sheet use the same words and support is not translating between them.
- The first failing launch in the launch log, which is usually earlier than the first ticket and is the figure you will be asked for.
- Whether callbacks are failing as well as launches. Only the second case leaves open rounds at the studio.
- The exported list of open round ids with their stakes, taken once at the start, and the count on the sheet.
- The threshold that triggered the flag and the first successful launch that will clear it, so the flag comes off on evidence.
- Who cleared the flag, at what time, and the version of the message players were shown while it was on.
FAQ
Questions operators ask
Should we automatically enable a different studio's similar game?
No. A substitute under the same tile misleads the player. They can choose another game from the lobby that is still up.
Should the lobby hide a studio after a single failed launch?
No. One timeout can be a player's network. Hide the studio when launches for it are failing as a pattern across players, against a threshold you set before the incident. Restore the tiles when a real launch succeeds, not when a status page looks hopeful.
Can we point the dead tiles at another studio's similar game?
No. The player chose that tile. A different game behind it is a mis-sell. They can pick something else that is still launching. The original tile stays unavailable until that studio is back.
Do we flag the studio on every brand, or only the brand whose launches are failing?
A studio incident is not brand-specific, because the same engineers serve every brand on the integration. Flag it everywhere and record which brands were actually affected. A brand left live during the incident generates the open rounds the rest of the estate then spends the evening closing.
One title from the studio fails while the others launch. What do we hide?
The title, if the catalogue can express that, and say so in the banner. Taking a whole studio down removes tables players are enjoying. Where the catalogue only has a studio switch, take the studio down and note the reason on the sheet, rather than leaving a quarter of the tiles failing quietly.
Keep reading
Related pages
- StudiosStudios300+ studios across slots, live casino and instant games, switched on per licence rather than as a blanket. A name here is not a promise.Read
- Guide · WalletVoids and rollbacksWhen a studio voids a round, the rollback returns exactly the stake the debit took, keyed to the original transaction id.Read
- Guide · SupportStuck roundsLeave a frozen spin pending until you know whether the credit is in flight, voided or genuinely stuck, then escalate with the id.Read
- Product · Game aggregationAggregatorOne aggregator contract for licensed operators: 40,000+ games from 300+ studios at 13% of GGR in Asia and 16% elsewhere.Read
Get a written quote and sandbox access
Tell us your licence, target markets and products. You get the GGR rate in writing, the studio list for your markets and sandbox credentials.