Status Updates

DAEDALUS

Wallet

This week the team continued working on the upcoming Daedalus release for the Shelley Testnet, which will include delegation features and the ability to check your reward balance from ITN wallets. The team also implemented a new password strength meter feature, and many other smaller UI/UX improvements.

ADRESTIA

This week, the team reviewed the Docker image to make it minimal & self-contained, using static binaries produced by CI. The introduction of dynamic control parameters prompted the team to do some database clean-up and housekeeping work. The team revised API specifications regarding the listing of stake pools. The team considered keeping an identical API between the ITN and the HTN, but there are significant differences between Jörmungandr and cardano-node with regards to stake pools, so the API had to be changed. One endpoint (listing stake pools) needed to be changed. See this URL for more information.

The key changes are:

1. Stake pools no longer return an apparent_performance metric.
2. Stake pools are no longer ordered by desirability. Instead, they are ordered by non_myopic_member_rewards.
3. Because of change #2, the endpoint now requires an explicit stake amount to be passed as a query parameter to indicate the intended stake to delegate.
4. Metadata no longer contains a pledge_address, but does now contain an explicit pledge in lovelace.

The team discovered that the Get reward account balance from the local state query protocol request was causing problems. The request is relatively slow on the node's end, and tends to fail when switching from one epoch to another. The team is now working on installing some caching to get a more consistent behavior.

The team also continued working on debugging the integration test cluster. Extra loggers were installed to pinpoint issues when they occur. Stake pools were re-enabled in the integration cluster, which was necessary to observe specific things, such as rewards growing.

The team fixed an oversight in the endpoint that returns protocol parameters. The endpoint was returning parameters from genesis only, instead of refreshing them regularly using the local state query protocol.

The team monitored the chain to discover, parse, and extract stake pool registrations and the various details contained in registration certificates.

Work was also done on finalizing the listing of stake pools using the local state query protocol. Pools can now be listed and ordered based on a given indented stake amount. A few outstanding issues remain: Wiring together metadata, pools data from the local state query, and pools data discovered from the chain (cost, margin, etc.)

Finally, the team worked on a feature to automatically and asynchronously fetch metadata associated with stake-pools from remote servers. This is an interim solution until the stake pools metadata aggregation server (SMASH) is able to discover and serve metadata itself. Fetching is done one-by-one, with a strict timeout and a defensive client to prevent bad actors from denying cardano-wallet. The team also enabled the submission of already signed and serialized Shelley and Byron transactions through cardano-wallet.

NETWORKING

The team updated the maximum ingress queue limits as they were too low for the Shelley testnet. They also implemented randomized chain-sync timeouts to avoid the simultaneous failure of peers disconnecting from each other if a block was not produced for sufficiently long.

The team is also making progress with enabling ΔQ measurements, and have an early integration with block-fetch logic. This will make the block fetch decisions from whom to download a block robust. Block fetch will be able to adjust itself to changing network conditions.

In addition, the team finished updates that are needed to restart mini-protocols and are integrating this with our new server implementation (based on a connection manager). We also wrote connection-manager tests: using QuickCheck and our simulation monad io-sim as well as tests which run uni- and bi- directional TCP connections.

Finally, the team carried out another review session of our network design documentation.

DEVOPS

This week the team migrated shelley_testnet from version 1.14.0 to 1.14.1, and tested the MIR process to register ITN rewards on shelley_testnet. It's important to highlight that the team managed to restore everyone's rewards from ITN on shelley_testnet, which represents a huge milestone. The team would like to thank everyone involved in working with jormungandr, cardano-node, and cardano-wallet for making this possible, and so simple for DEVOPS to implement.

The team deployed version 0.9 of ITN and submitted a plan for the community to vote on whether or not to extend ITN rewards. Also, the team worked on integrating a local test cluster for fast epochs (minutes), usable for quick testing of the Daedalus integration.

Finally, the team made available a db-sync tool to extract relays and generate a public topology.json on explorer here. The team recommends to pick one or two from the list, rather than using them all on the same relay.

CARDANO DECENTRALIZATION

This week the team completed the historical stake pool performance calculations for both non-myopic rewards and stake pool rankings. An approximation of a probability distribution (a histogram), is now stored in the chain state, so that samples can be obtained. We will be updating the formal specification with these details shortly.

The team also worked hard to put a considerable amount of benchmark tests in place, and have used them to fix several performance issues.

In addition, they worked on Byron address support in the ledger and finally, completed the VRF implementation which now offers some testnet performance improvements.

GOGUEN

This week the Plutus team worked on providing the ability to create dynamic slot ranges for users who are writing smart contracts. These slot ranges can control what contracts execute, for example, a user could create a ‘Time To Live’ field for a contract, so that it executes if something has not happened in the last 10 slots from the current time.

The team also applied a fix for potential runtime errors when the blockchain rollup cannot find a reference. Finally, the team added 'Examples.Data.Vec’ in Plutus Core to show how Church- and Scott-encoded vectors are represented.

The Marlowe team added a first version of a new wallet simulation tab to the Marlowe Playground. This tab enables users to see what features a wallet needs so that it can run Marlowe contracts and how a contract behaves between multiple wallets.

WEEKLY DEVELOPMENT REPORT

DAEDALUS

Wallet

This week the team focused on the first Daedalus release for the Shelley Testnet. Work on the implementation of the hardware wallet support is nearly complete. The team also worked on bug fixes and UI/UX improvements.

ADRESTIA

This week, the Adrestia team added new features for cardano-wallet-itn, cardano-wallet-byron, and cardano-wallet-shelley on all three platforms (Linux, MacOS, Windows), plus corresponding docker images. They had to abandon the Haddock documentation export (an insignificant loss) to enable CI to run and build all these new features within an acceptable timespan.

The team also worked on getting an integration test cluster ready by running a BFT leader and stake pools. Networking issues occurred, particularly affecting CI, which makes the cluster inefficient at the moment. This pioneering work did however allow the team to continue integration tests of various API scenarios.

The team also worked on many other features: For example, Join/Quit stake pools endpoints are now available on cardano-wallet-shelley to delegate funds from cardano-wallet. Delegation fee estimation endpoint are also available now on cardano-wallet-shelley.

The Adrestia team reworked the stake pool model in the API, so cardano-wallet can serve both Jörmungandr & cardano-node stake pools. The model has some subtle differences (cardano-node gives us non-myopic member rewards, for example, while Jörmungandr used the desirability model. Also, there currently is no access to the apparent performance with cardano-node. The team also drafted initial work for reporting correct reward account balance in cardano-wallet-shelley.

And finally, the team worked on getting final user-facing encoding done at the API level for Shelley, so Byron addresses are served Base58-encoded (Shelley ones are bech32). Right now, everything is hexadecimal to match what cardano-cli produces.

NETWORKING

This week the network team pinpointed the source of the re-occurring network-wide disconnects in the F&F testnet, and fixed the problem. The team also made good progress on features required for the P2P deployment.

DEVOPS

This week, the DevOps team began working on a local multi-node cluster controlled by supervisord. The team also worked on a Haskell tool to generate a genesis file from Byron snapshot, and also worked on packaging sentry monitoring tool, and updated the graphql-engine to version 1.2.1, to add support for cardano-graphql/explorer.)

CARDANO DECENTRALIZATION

Over the past week, the team updated the formal specification with changes made to the exec specification (MIR can now take from the treasury.) Also, the team did a lot of benchmarking using the criterion library, which led to performance optimizations to the old Byron relation class. These optimizations will be announced next week.

Property tests are now much more aligned with recent changes in the exec spec, though the team is still working on achieving non-zero min value UTXO and non-zero min pool cost.)

The team also added a min pool cost protocol parameter, and a check for it. Also, the team continued their support for integration efforts, and worked intensively on the apparent pool performance, calculating and storing the relevant data to the ledger state. This is still a work in progress, however.

GOGUEN

This week the Plutus team made improvements to the cabal build system, including pinning the index state so a freeze file is no longer required, getting the index state from the cabal file for Nix usage, and removing a rogue source dependency that was causing an issue.

The team also began working on Actus integration and produced a cost model for Plutus. They also made some updates to the client and server node protocols, and some edits to the comment syntax.

MARLOWE

The Marlowe team worked on the display of tokens as ada in the Marlowe Playground, made some updates to the Nix process, and worked on the arithmetics of multiplication/division.

WEEKLY DEVELOPMENT REPORT

Daedalus

Wallet

There is no update from the team this week.

Cardano Explorer

There is no update from the team this week.

Adrestia

This week the team finalized the Shelley "basic" transaction layer (UTXO payments) and performed extensive testing of it. During this process, the team discovered that "genesis stake pools" are not a feature of Jörmungandr. Leaders registered in the genesis file are not pools, they are BFT leaders. Because of this, most of the integration testing so far has happened on BFT Shelley leaders only. Setting up pools for integration testing will involve a lot of additional work.

The team also did preparation work for setting up the test integration to get a cluster of stake pools running. A lot was achieved, but pools need to be registered and have funds delegated to them, which means that three separate certificates must be created, and there must also be funds available for the transactions.

During this week, the team created the initial implementation of delegation in Shelley, although this implementation cannot be tested properly until work on the test cluster is completed.

The team cleaned up protocol parameters, and the decentralization level (d parameter) now reports from live data (this parameter had been hardwired up to this point.) But now, the d parameter is properly fetched from a running local node, using a local state query protocol.

This week, the team improved the wallet integration to work with cardano-node-1.13.1, and worked on several fixes too: #1701, for example, where existing Shelley wallets would wrongly report as having no passphrase. A missing database migration was the cause, which has now been resolved. Also, the team improved Hydra evaluation time by about 30%, making CI a bit easier.

The team enabled restoration from extended account public key on Byron sequential wallets (Icarus), to enable testing hardware wallet integration on Byron during the development of Shelley hardware applications.

Finally, the team approached the limits of Travis' free tier, which meant that some releases had to be postponed. The team is working on finding a feasible and budget-friendly solution to compile multiple versions of cardano-wallet (Jörmungandr, Byron, and Shelley) going forward.

Networking

This week, the networking team finished implementing a KeepAlive mini-protocol. Sending a 'keep alive' message is required by various deployment scenarios to keep network connections open. The team used the design of another warm peer protocol that allows to sample chain tips. They also merged extended tests of a bi-directional, on-demand multiplexer, and made progress in reviewing existing documentation.

DevOps

This week the team supported the Shelley Public Testnet by adding and testing new features. The faucet was given new features and enhanced functionality, for example, and the team worked on Rust cross-compilation to Windows with nix.

During this week, the team tested MIR certificates and ITN stake key conversion and registration features. And finally, the team upgraded repos to build with nixos 20.03, and finalized cardano-graphql hasura update changes.

Cardano Decentralization

The implementation of the Byron-to-Shelley hard fork is nearing completion, so thorough testing will commence soon.

This week the team worked on bug fixes to the nixified benchmarking pipeline and added a set of new metrics relevant for benchmarking and stake pool operators to TUI and RTview. An anomaly in the benchmarking in component "cardano-db-sync" was analysed and reported to the dev team, and the team made a slight change to how chain selection works: forks will now be resolved in favour of the block issuer with the lower leader value, which should slightly reduce the likelihood of long forks.

.

Goguen

This week the Plutus team fixed some problems in the plutus-scb (smart contract backend); an issue with the chain index startup sequence, and a general Haddock issue. The team also made some updates to use special QuickCheck operators, which resulted in better error messages. Other updates were made that increased the Plutus Playground load balancer timeout, and some simplifications were made to the FormSchema for contracts.

The Marlowe team investigated how to write assertions in Marlowe contracts, so that they can be checked through static analysis.

WEEKLY DEVELOPMENT REPORT

Daedalus

Wallet

This week the Daedalus team continued work on the integration of both the hardware wallet and the Shelley Cardano node, as well as preparing for the upcoming Daedalus 2.4.0-ITN1 release.

Cardano Explorer

There is no update from the team this week.

Adrestia

This week the team added a migration endpoint to the Shelley component of the API, to make it easy to migrate a Shelley wallet to another wallet.

The version and integration with cardano-node was bumped following the pioneer releases. Nix script improvements were also made to improve access to various cardano-node deployment information.

Revisions were made to the getNetworkParameters endpoint to remove the extraneous "epochId" path parameter and to add a "decentralizationLevel" indicator to the set of returned parameters.

Work continued on support for cardano-node. The wallet can now send and receive basic UTxO transactions, despite some challenges regarding fee calculations. The majority of integration tests are now passing, and we are continuing to fix the remaining tests one-by-one. In the interim, we are adding support for delegation certificates and stake pool listing. Support was also added for cardano-node in Shelley mode, when connected to cardano-wallet-shelley.

Some minor API documentation improvements and fixes were made, using feedback gathered from exchanges as well as from users who raised questions on Slack. In addition, the package delivery workflow was automated to reduce the number of steps post-release.

A user friendly command-line interface was delivered for performing HD derivation in the console for all types of Cardano wallets. The associated commands are simple and well integrated, and also include multiple output encoding as well as good error handling. For example, a user can create a recovery phrase and easily obtain the public key of the associated reward account.

Issue #1670 was also fixed, where input resolution was not carried out over the entire database, but only on the fetched data. Additionally, a bug regarding transaction ordering was fixed.

Networking

This week, the networking team merged and reviewed multiplexer changes which allow on-demand start of protocols. They also continued working on a set of comprehensive tests which will cover scenarios needed by the p2p-governor.

The team also implemented a server that is based on the new connection-manager, but this has not yet been merged to the master branch.

Investigations took place on how MonadDelay is using unbounded delays in a safe way (this is especially useful in 32-bit architectures, where previously maximal delay was about 30 minutes). In addition, the team updated cardano-db-sync to use the cardano-client library, and worked on updating other clients as well.

Discussions were held about the warm-peer protocol, which is needed to send keep-alive messages and query blockchain tip information. Last but not least, the team has been busy preparing their design document for a public release.

DevOps

The team tested ITN rewards transfer CLI tooling and update proposals. They also made Daedalus improvements to support the Shelley public testnet as well as faucet improvements to support different levels of API keys. Finally, they worked on Rust cross-compilation tooling and release builds for musl/mac.

Cardano Decentralization

The team successfully tested the hard fork combinator by simulating a hard fork between two mock ledgers. They also added the change discussed at the CSM regarding how to give delegators time to react when stake pools change their parameters. Stake pool re-registration certificates now always delay updating the stake pool parameters until the beginning of the next epoch, and crucially, after the stake distribution snapshot.

Genesis delegation certificates now register their VRF keys (as a hash). The team also ensured that the VRF key in the block header is checked against the registered hash for PBFT blocks, and the two VRF values (nonce and leader), are checked for PBFT blocks.

Some tests were moved from consensus to cardano-ledger-specs and the spec was also updated with all the recent activity on the exec model.

Finally, some investigation work took place on the Byron addresses to figure out how best to handle the authorization of Byron addresses in Shelley.

Goguen

This week the Plutus team made some organizational changes by merging plutus-emulator into plutus-contract and plutus-contract-tasty back into plutus-contract. They also split out the demo for the plutus-scb (smart contract backend), to provide two contract choices. Finally, they updated the sequence of handleBlockchainEvents and improved some ambiguous syntax for the builtin application.

The Marlowe team spent time testing the changes made in the previous week to the Marlowe Playground. They also made further changes to the Isabelle build task for NIX/Hydra and also worked on the Isabelle build task for NIX/Hercules.