TL;DR

We all were using the wrong approach to getting balance snapshots which led to incorrect Tezos staking balances on some cycles and therefore incorrect delegator shares and incorrect staking reward payouts.
TzKT indexer by Baking Bad has solved this problem.

While developing the TzKT indexer (an open source Tezos blockchain indexer behind the TzKT explorer), we found out a lot of interesting things about Tezos and how it actually works. Now we are going to gradually share our experience and spread our knowledge in order to help other Tezos developers avoid our mistakes and to make all tezosians more literate in general.

In this article we will bring some light on the balance snapshots, especially on staking balances and total rolls, which, as far as we know, no one was calculating correctly so far. We won’t go into what a balance snapshot is and why it is needed. We hope you already know that. Otherwise, start reading from this article.

Our goal is to demonstrate what you shouldn’t do when calculating a staking balance or total rolls in a particular snapshot. All the code below is hosted on JSFiddle, so you can open it, run and see the results in real-time.

How can we determine the block where the snapshot was taken for cycle N?

First of all, we use the specific Tezos RPC endpoint to get a snapshot index and then, using this index, find a snapshot block.

let get = async path =>
  await (await fetch('https://mainnet-tezos.giganode.io' + path)).json();
  
const CYCLE_SIZE = 4096;
const PRESERVED_CYCLES = 5;
const SNAPSHOT_BLOCKS = 256;

(async function snapshot() {
  let cycle = 220;
  let level = cycle * CYCLE_SIZE + 1;
  
  let snapshotIndex =
    await get(`/chains/main/blocks/${level}/context/raw/json/cycle/${cycle}/roll_snapshot`);
    
  let snapshotBlock =
    (cycle - PRESERVED_CYCLES - 2) * CYCLE_SIZE + (snapshotIndex + 1) * SNAPSHOT_BLOCKS;
  
  console.log(`Cycle: ${cycle}`);                     // 220
  console.log(`Snapshot index: ${snapshotIndex}`);    // 15
  console.log(`Snapshot block: ${snapshotBlock}`);    // 876544
})()

As we can see, for the cycle 220, the level of the snapshot block is 876,544.

How we were getting Tezos staking balances before TzKT

We just used the specific RPC endpoint to request the staking balance of the particular delegator from the context of a particular block.

let get = async path =>
  await (await fetch('https://mainnet-tezos.giganode.io' + path)).json();
  
const ROLL_SIZE = 8000000000;
  
(async function stakingBalanceSnapshot() {
  let level = 876544;
  let address = 'tz1W5VkdB5s7ENMESVBtwyt9kyvLqPcUczRT';
  
  let stakingBalance =
    await get(`/chains/main/blocks/${level}/context/delegates/${address}/staking_balance`);
    
  let rolls = Math.floor(+stakingBalance / ROLL_SIZE);

  console.log(`Staking balance: ${stakingBalance}`);   // 7257772535921
  console.log(`Rolls: ${rolls}`);                      // 907
})()

As we can see, this baker has 7,257,772.535921 XTZ staking balance, it’s 907 rolls. This seems correct because RPC indeed returns the correct staking balance at the end of the specified block.

How we were getting total rolls before TzKT

Roughly, we got all active bakers from the context of a particular block and then summarized their rolls calculated from their staking balances.

let get = async path =>
  await (await fetch('https://mainnet-tezos.giganode.io' + path)).json();
  
const ROLL_SIZE = 8000000000;
  
(async function totalRollsSnapshot() {
  let level = 876544;
  let delegates = await get(`/chains/main/blocks/${level}/context/delegates?active=true`);  
  let totalRolls = 0;
  
  for (let [i, address] of delegates.entries()) {
    console.log(`${i} of ${delegates.length}`);
    
    let stakingBalance = 
      await get(`/chains/main/blocks/${level}/context/delegates/${address}/staking_balance`);

    totalRolls += Math.floor(+stakingBalance / ROLL_SIZE);
  }
  
  console.log(`Total rolls: ${totalRolls}`);    // 80537
})()

So, there are 80,537 rolls in total. This also seems correct, because RPC indeed returns the relevant list of active bakers and correct staking balances at the end of the specified block.

Was that correct?

Well, no 😁. But let’s better check it ourselves using another RPC endpoint for accessing all rolls and all roll owners in a particular snapshot.

let get = async path =>
  await (await fetch('https://mainnet-tezos.giganode.io' + path)).json();
  
(async function rollOwners() {
  let cycle = 220;
  let cycleLevel = 901121;
  let snapshotIndex = 15;
  
  let delegate = 'tz1W5VkdB5s7ENMESVBtwyt9kyvLqPcUczRT';
  let pubKey = 'edpktpTCe72ZvkzBTELDbaiUJit7VgzQpzmAHw6XUhy6Tre83mPh4Z';
  
  let rollOwners = // request takes ~30 seconds
    await get(`/chains/main/blocks/${cycleLevel}` + 
              `/context/raw/json/rolls/owner/snapshot/${cycle}/${snapshotIndex}?depth=1`);

  let delegateRolls = rollOwners.filter(x => x[1] === pubKey).length;
  let totalRolls = rollOwners.length;
    
  console.log(`Delegate rolls: ${delegateRolls}`);    // 906
  console.log(`Total rolls: ${totalRolls}`);          // 81261
})()

As you can see, this baker has actually 906 (not 907) rolls and total rolls are 81,261 (not 80,537). Now, let’s see what TzKT API does return:

Total rolls: /v1/cycles/220

Total rolls: 81261

Baker’s rolls: /v1/rewards/bakers/tz1W5Vkd…qPcUczRT/220

Staking balance: 7254007681786
Baker's rolls: 7254007681786 / 8000000000 = 906

That’s correct!

Why is RPC-approach wrong?

When you get account balance from the context via RPC, you get the balance at the end of the specified block. As we discovered, snapshots are taken not at the end of a particular block, but a little earlier, so that some balance updates in the block are not committed yet. In other words, balance snapshot and balance at the end of the snapshot block are not the same things.

This actually affects only specific cycles. As you can see, in most cycles there are no visible differences between RPC-approach and TzKT API. If you take a closer look you will see, that the problem takes place only in cycles where a snapshot index = 15, which corresponds to the last block in the cycle. As you may have already guessed, the problem is likely in the frozen cycle rewards. Yes, snapshots are taken before the cycle rewards are unfrozen, so bakers have lower balance and lower staking balance (because frozen rewards are not included). That’s why we see above that the baker has fewer rolls in the snapshot than at the end of the block where the snapshot was taken.

IMPORTANT

If you are using RPC to get staking balance snapshots, you should take the above into account in order to get correct blockchain data.

Anyway, you can always use TzKT and not worry about data correctness, because it knows Tezos from “tz” to “KT” 😺.

Consequences of using incorrect balance snapshots

First of all, using incorrect balance snapshots leads to incorrect delegator shares (because baker’s share becomes slightly larger), which leads to incorrect rewards distribution. On average, delegators will receive approximately 0.05% less rewards than expected.

Also, if you somehow use snapshot data (for example, publish Tezos stats like “staking ratio” or “total in staking” or calculate “luck ratio” based on total rolls, etc.), we highly recommend you check the correctness of the data you use. Statistics based on wrong data don’t make any sense as well as code processing incorrect data will return worthless result.

What about Baking Bad?

Since Baking Bad has switched to TzKT API (because it shows the most accurate and detailed Tezos data) you can see that some of historical payouts have become inaccurate, even though before they were displayed as accurate:

NOTE

This definitely won’t affect your rating, because we completely understand that this happened because we all were using the same incorrect data and it’s really no one’s fault. There were just no reliable data sources 😔.

Also, we’ll try to find a workaround and somehow mark these payouts to not mislead users, who may mistakenly think that their bakers don’t know how to pay Tezos staking rewards accurately.

But now, when people are aware of the problem and there is a solution, we believe Tezos bakers will fix their payment tools in the near future.

Conclusion

Don’t try to get snapshotted balances (as well as staking balances) from the context via RPC endpoints, unless you know what you do.

KEEP IN MIND

Balance snapshot and balance in the snapshot block context (at the end of the snapshot block) are not the same!

Feel free to use TzKT API because it shows the correct values. We believe that data quality comes first, especially when we talk about a blockchain API.

Contact us if you have any questions.

Cheers 🍺