Today we released TzKT v1.4, another version of the free open-source Tezos blockchain indexer and API with several important features, enabling more advanced use cases and making developers' life much easier.
Let's walk through the changes, but first of all, pay attention to the breaking changes.
# Breaking bad changes
# For API users:
parametersfield in transactions has been deprecated and will be removed in the next major release. Use theparameterfield instead, and use query parameters?entrypoint=and?parameter=for filtering.
We tried to smooth this change and added a kind of backward compatibility for most common use cases, so nothing should break actually. However, we strongly recommend you update your apps as soon as possible.
The most relevant API documentation is always available on api.tzkt.io. Feel free to reach us in our telegram chat or in Tezos dev slack with any questions.
# For database users:
Parameterscolumn has been dropped from theTransactionOpstable. UseEntrypoint,RawParametersandJsonParameterscolumns instead;Kindcolumn in theAccountstable has been extended with another possible value2- asset, in addition to1- smart contract and0- delegator contract.
# For self-hosted:
TzKT has migrated to the newer .NET 5.0 so you need to install .NET 5.0 instead of .NET Core 3.1. Check out the installation instruction on GitHub.
# Developer-friendly smart contract data
Smart contracts in Tezos are very flexible, which is great because it allows developers to do the same things in different ways. On the other hand, it makes life much more difficult for end users because of the ambiguity of the data.
For example, these are three equal ways to specify transaction parameter to call the initiate entrypoint in the Atomex smart contract:
%default Left Left Pair (...);%fund Left Pair (...);%initiate Pair (...).
Another example, these are three equal values, representing the same Tezos address:
0x01e2cf37d9997a1cf9b0995394264c289df016996a00;KT1VG2WtYdSWz5E7chTeAdDPZNy2MpP8pTfL%default;KT1VG2WtYdSWz5E7chTeAdDPZNy2MpP8pTfL;
It's really hard to work with data that can be represented by completely different values and which also should be parsed/handled differently, especially if you are new to Tezos and simply do not know all the specifics.
Thus, we have taken a big step toward making Tezos blockchain data (and, in particular, smart contract data) more user-friendly.
# Normalized JSON parameters
Starting from v1.4 the TzKT indexer automatically normalizes transaction parameters so that there is no more ambiguity.
Looking at the examples above, it no longer matters how users pass parameters: with %default or %initiate entrypoint,
in optimized 0x01e2cf3... or readable KT1VG2W... form. TzKT automatically parses, analyzes and converts it to a single readable form.
Then, TzKT transforms Micheline to human-readable JSON, using annotations from contract scripts, and it's like a breath of fresh air for any developer. Just imagine you are working not with this:
{
...
{
"prim":"Right",
"args":[
{
"prim":"Left",
"args":[
{
"prim":"Pair",
"args":[
{
"bytes":"00009472982d7f6b096bc57d6da95e0b8ec8ee37e72f"
},
{
"prim":"Pair",
"args":[
{
"bytes":"0000bf97f5f1dbfd6ada0cf986d0a812f1bf0a572abc"
},
{
"int":"10000"
}
]
}
]
}
]
}
]
}
...
}
but with this:
{
"entrypoint":"transfer",
"value":{
"from":"tz1ZAwyfujwED4yUhQAtc1eqm4gW5u2Xiw77",
"to":"tz1d75oB6T4zUMexzkr5WscGktZ1Nss1JrT7",
"value":"10000"
}
}
For more flexibility we also added the ?micheline= query parameter to configure the format of the transaction parameters. The following values are allowed:
0orjson- human-readable JSON (deserialized) (example);1orjsonString- human-readable JSON string (example);2orraw- raw Micheline (deserialized) (example);3orrawString- raw Micheline string (example).
NOTE
Both rawString and jsonString are very useful when you do strongly typed deserialization and you don't know the type of the parameter
until you read the entrypoint value. So you can first deserialize the transaction, then read the entrypoint value and then, depending on the entrypoint,
deserialize the parameter string to a specific type (it's a kind of lazy deserialization).
# Filter transactions by parameter like a boss
Working with normalized JSON parameters is surely great, but what would be even better is the ability to request transactions with specific parameters, and TzKT pushes the envelope.
In addition to the wildcard pattern (the .as query parameter mode, available in earlier versions),
we've added the "json" pattern, that allows you to specify the path to a particular field within the JSON parameter
to filter by. It looks like this: ?parameter[.path][.mode]=....
A few examples to make you realize it:
- How to get all incoming kUSD transfers for the account
tz123...? Easily:parameter.to=tz123...(example). - How to get all Dexter XTZ to USDtz trades with the amount > 1k? Easily:
parameter.minTokensBought.gt=1000000000(example). - How to get Atomex atomic swaps with the refund time in Feb 2021? Easily:
parameter.settings.refund_time.as=2021-02-*(example).
Also, now you can use the ?entrypoint= query parameter to filter transactions by entrypoint. A few examples:
- How to get all Dexter wXTZ/XTZ trades? Easily:
entrypoint.in=xtzToToken,tokenToXtz,tokenToToken(example). - How to get all operations with tzBTC related to a specific account? Easily:
entrypoint.in=mint,transfer,burn¶meter.as=*tz123*(example).
# Build Micheline parameters from JSON parameters
We’ve added an endpoint for building Micheline parameters from its JSON representation. All you need is to specify a smart contract, an entrypoint, and the JSON value you want to convert:
{
"from": "KT1PWx2mnDueood7fEmfbBDKx1D9BAnnXitn",
"to": "KT1PWx2mnDueood7fEmfbBDKx1D9BAnnXitn",
"value": "1234"
}
and you will get raw Micheline that can be attached to a transaction and then forged and broadcasted to the blockchain:
{
"prim": "Pair",
"args": [
{
"bytes": "01a3d0f58d8964bd1b37fb0a0c197b38cf46608d4900"
},
{
"prim": "Pair",
"args": [
{
"bytes": "01a3d0f58d8964bd1b37fb0a0c197b38cf46608d4900"
},
{
"int": "1234"
}
]
}
]
}
Of course, to convert JSON to Micheline you need to know the format (schema) of the parameter to provide a valid JSON value. We have added this information to the tzkt.io explorer, so that you can easily find schemas for all entrypoints and all smart contracts.
In fact, such things are also available in popular Tezos SDKs like Taquito, PyTezos, etc., but they obviously do not cover all languages, but the HTTP API does, so those developers who don’t use these SDKs will be able to interact with smart contracts in a convenient way, using the API.
TzKT will eventually be supplemented with more such tools, so that it won’t be only a data API, but also a toolset for developers.
# Normalized JSON storage
We’ve added indexing of contract storages, which are automatically normalized and available in both raw Micheline and human-readable JSON formats, as well as transaction parameters.
The following API endpoints for accessing contract storages have been added:
- /storage - returns storage value in human-readable JSON;
- /storage/schema - returns JSON schema of the storage;
- /storage/raw - returns storage value in Micheline format;
- /storage/raw/schema - returns Michelson type of the storage;
NOTE
Only simple storage is indexed at the moment. Lazy storage (aka bigmaps) indexing is on the way.
# Get specific fields from the storage
Similar to how you can filter transactions by parameter, specifying the path to the field to filter by, you can use ?path= query parameter to specify the path to the specific field in the storage that you need to receive.
For example, imagine, the contract we are working with has the following storage:
{
"vesting":{
"vesting_schedule":{
"next_payout":"2019-09-29T08:30:00Z",
"payout_interval":"2629800"
},
"vesting_quantities":{
"vested_balance":"162775594954",
"vesting_increment":"199041301565"
}
},
"key_info":{
"key_groups":[
{
"signatories":[
"edpkvS5QFv7KRGfa3b87gg9DBpxSm3NpSwnjhUjNBQrRUUR66F7C9g",
"edpktm3zeGMzfzFuqgyYftt7uNyVRANTjrJCdU7bURwgGb9bRZwmJq",
"edpkucCnbeGPWNvGHeTQ5hENHPrc8txBBiQXNphu3jgv9KYbhQBovd",
"edpkuNjKKT48xBoT5asPrWdmuM1Yw8D93MwgFgVvtca8jb5pstzaCh"
],
"group_threshold":"2"
},
{
"signatories":[
"edpkukogJzCZSAfc5foRpEZGsryMjiXj77VxcFLzZoreB1vZWFeKc2",
"edpkuSZ1GoM6MALh4fPZBrwDhGwY9vENEoyctcXuDK3yoiX4xWhMaA",
"edpkvXL9B32DcEbMNiMangcSFMvAd8NBwH8AfmRb6iBbHxLgx3J59P",
"edpkumCFhgS94cjZXiFnnq7MFsaWBz4Tp78AX2fZfhB9J9hcgcKxgy"
],
"group_threshold":"2"
},
...
],
"overall_threshold":"4"
},
"pour_info":{
"pour_dest":"tz3bTdwZinP8U1JmSweNzVKhmwafqWmFWRfk",
"pour_authorizer":"edpkteDwHwoNPB18tKToFKeSCykvr1ExnoMV5nawTJy9Y9nLTfQ541"
},
"replay_counter":"6505"
}
But we want to receive not the whole storage, but only the first signatory from the second key group.
To do this we can use the ?path= query parameter to tell the API that we need only a specific part of the storage:
{contract}/storage?path=key_info.key_groups.2.signatories.1
edpktjDaR4gmEZNSnnuSWMmJ4fhYmMD4Lk1WkDCfWRWdbGHQ9eNT45
This is very useful when the storage is rather large and we don’t want to waste resources on transferring and deserializing redundant data.
# Get storage at any time in the past
Also, you can easily access contract storage value at any time in the past.
If you want to get the contract storage at a specific level (at the end of the block), you can use the ?level= query parameter:
{contract}/storage/?level=1354679
If you want to get the contract storage after a specific operation, you need to request that operation and the resulting storage value will be included in the response: /transactions/{hash}
# Get storage history
In addition, we’ve added an API endpoint to get contract storage history, including information about the operation, caused the changes, and the resulting storage value: {contract}/storage/history
# Contract Code
Starting from v1.4 the TzKT indexer stores all contract scripts, including parameter type, storage type and contract code.
It’s worth noting that we have developed our own binary format that is used to store raw Micheline, which is x10 times more compact than Micheline JSON and 25% more compact and much faster than forging Micheline bytes.
So, contract scripts are stored in binary format, however, there is an API endpoint that allows you to get values, deserialized to the desired format:
- /code?format=0 - returns contract code in Micheline format;
- /code?format=1 - returns contract code in pure Michelson;
- /code?format=2 - returns contract code in TzKT’s binary format.
# WebSocket API
Another great feature in TzKT v1.4 is the WebSocket API, which enables you to subscribe to real-time Tezos data, so that you can receive the latest data as soon as it appears on the node and the TzKT indexer processes it. For interactive applications, that require real-time functionality, TzKT WebSocket API is a much better solution than spamming the REST API every second, checking for new data.
TzKT WebSocket API is designed to be very reliable, so if you properly handle all the messages from the API be sure you won’t miss any data nor receive duplicated data. All supported message types are explained in the docs, so, please, be sure to read it first.
NOTE
If you are working on critical apps you may ask us for some code review regarding usage of our services to ensure you are doing it right and not making obvious mistakes.
We started with a very basic set of subscriptions:
- new head - getting indexer’s head every time it’s changed;
- new block - subscribing to new blocks;
- new operation - subscribing to operations of the selected types;
- new account operation - subscribing to operations of the selected types and related to the specified accounts only.
In the near future we will add more events and subscriptions with more data, so if you have any feature requests, this is the best time to apply for it (just open an issue on GitHub).
# Simple JavaScript client
TzKT WebSocket API is based on SignalR, which is actually great because there are a lot of already existing client libraries for many languages (C++, C#, Java, TypeScript/JavaScript, Python, etc.). In addition, you don't need to be a super-experienced developer or have a complete understanding of WebSocket internals. SignalR does all the complicated stuff for you and is therefore really easy to use.
Here is how a simple JavaScript TzKT WebSocket client looks:
const connection = new signalR.HubConnectionBuilder()
.withUrl("https://api.tzkt.io/v1/events")
.build();
async function init() {
// open connection
await connection.start();
// subscribe to head
await connection.invoke("SubscribeToHead");
// subscribe to account transactions
await connection.invoke("SubscribeToOperations", {
address: 'KT19kgnqC5VWoxktLRdRUERbyUPku9YioE8W',
types: 'transaction'
});
};
// auto-reconnect
connection.onclose(init);
connection.on("head", (msg) => {
console.log(msg);
});
connection.on("operations", (msg) => {
console.log(msg);
});
init();
# Other improvements
In addition to described features, we made several internal optimizations, increasing indexer and API reliability. Also, we’ve upgraded core dependencies, because it would be a sacrilege to not use state-of-the-art technologies in the most advanced blockchain in the world 😄
# .NET 5.0
We’ve migrated from the .NET Core 3.1 to .NET 5.0, the latest version of the free and open-source development platform with C# 9 and a lot of performance and security optimizations.
# PostgreSQL 13
Also, we’ve migrated from PostgreSQL 12 to PostgreSQL 13, the latest version of the world’s most advanced open-source database with significant improvements to its indexing and lookup system, and more.
As a result, we achieved faster response times and 25% reducing database size, so now it consumes only 16GB disk space for Tezos mainnet. A full database snapshot - only 2.4GB.
# Conclusion
v1.4 brings such important features as improved smart contract data and WebSocket API that makes TzKT even more advanced Tezos blockchain indexer and Tezos API. And there will be even more cool features in the future, so stay tuned.
We would like to remind you that TzKT is not a commercial product, but a community-driven project that is truly open-source and free for everyone and for any use.
Many thanks to all our friends and the entire Tezos community for helping with improving and adopting TzKT. You are awesome! ❤️
For the Tezos! 🍺