Status Updates

WEEKLY DEVELOPMENT REPORT

Daedalus

Wallet

This week the team was fully focused on the delivery of the rewards wallet version of Daedalus for the Incentivized Testnet. The team has been working on integration of the API endpoints, which are required for the delegation features, as well as updating the wallet UI to reflect the new functionality.

App Platform

There is no update this week, as the team was busy with the delivery of the rewards wallet version of Daedalus for the Incentivized Testnet.

Cardano Explorer

A block and epoch browser was created this week, enabling users to explore the blockchain starting from the tip, although it is compatible with providing a range of results via URL parameters. Navigation between the detail pages was completed, and various UX tweaks made to polish transitions and address issues raised during QA. The address introspection was improved to discriminate between address groups, and the application search logic refactored to better detect addresses, as well as simplify the API interaction to a single call when searching by ID. Finally, a performance issue with a particular query field was identified, and a fix implemented.

Wallet backend

This week the team spent some time fixing a few small bugs that escaped testing and internal QA as a result of the intense pace of work over the past few weeks. Now those are dealt with, the team is back on a more stable track for the final implementation of the remaining delegation features. The team is also aiming to improve the overall monitoring and usability of the software to make future debugging and support easier. Next week should see all the pieces finally fit together, giving birth to a new era of delegation and decentralization!

Networking

This week the team was focused on preparations for the 1.0.0 release of the Haskell node, which included merging error policies for the node and clients, as well as updating the cardano-node, cardano-byron-proxy, and cardano-explorer repositories. The hard work of the team also meant that they were able to include the DeltaQ estimator with tracing into the 1.0.0 release, and are working on reviewing and updating the documentation accordingly.

Elsewhere, the team has been making progress with the peer-to-peer governor, successfully running the first property-based tests. Two refactoring pull requests have also been prepared: the first to simplify version negotiation in the network-mux library, and the second to include timeouts and message size limits in the typed-protocol's driver.

DevOps

The team has made a lot of progress on the Byron rewrite side this week and will begin testing full cluster transition to the new codebase soon, using cardano-ops as the new deployment repository. Switching between iohk-ops and cardano-ops has been tested extensively, and the DevOps team will begin updating staging, testnet, and mainnet to the new codebase in the coming weeks.

Cardano Decentralization

The team has been very busy for the last few weeks, managing to rewrite the on-disk database format and get it finished in time for the 1.0.0 release (avoiding the need for a post-release migration), thanks to the extensive property tests that were already in place. The new format allows for faster validation by storing CRC32 checksums of blocks, faster reading of headers, which no longer requires parsing of the whole block, and more efficient streaming of blocks and headers straight from disk to the network without intermediary deserialization and serialization. Finally, a number of operations that required reading a whole block will now only need to read a handful of bytes instead.

Also this week, the team made changes to the transaction-level CBOR serialization, as well as adding a function to the Byron code so that the wallet backend can get updates without receiving the full UTXO. A lot of code was also migrated from Hedgehog to QuickCheck, following the decision to commit to using QuickCheck for all automated testing.

Goguen

This week the Plutus team made further updates to the multicurrency paper and started a restructuring initiative for the emulator to house elements in their own modules. They also updated the multisignature contract to use the new contracts API and added the capability to customize the signing process, as well as fixing an issue the Plutus Playground decoding. More generally, good progress is being made on integrating Plutus into the Cardano ledger rules thanks to cross-team collaboration efforts.

The Marlowe team fixed some issues that were appearing in the Marlowe Playground, including a problem with parsing of static analysis results for some outputs, and an anomaly where the amount of ada was not being updated correctly in the deposit suggestion.

WEEKLY DEVELOPMENT REPORT

Daedalus

Wallet

This week the team successfully delivered a new balance check version of the Daedalus wallet and continued preparing the next Daedalus release for the Incentivized Testnet, which will include delegation features.

App Platform

There is no update this week, as the team was focused on delivery of the new balance check Daedalus wallet.

Cardano Explorer

This week the team closed 34 issues, making progress in user experience, performance, test coverage, and implementation of new capabilities from Cardano GraphQL. The app now has near real-time data feeds without impacting the scalability of the Cardano GraphQL API service by using the network slot duration to poll, rather than a stateful connection over a WebSocket. Using an understanding of the network chain tip, deterministic information on the client now reduces demand on the userÅfs connection to a minimum, using a lightweight GraphQL query and field selection.

WALLET BACKEND

This week has been a very intensive period of fixes, polishing, and tweaking to deliver the best possible experience for users during the balance check phase of the Incentivized Testnet. It has been very demanding for the team, and they handled the challenge incredibly well, although it has necessarily delayed further work on delegation features. Nonetheless, this is still a very important milestone, as it is the result of more than a year of hard work to rebuild and extend the Byron wallets.

Finally, it's not without disappointment that the team discovered a significant problem that had been lurking for almost two years, resurfacing from as far back as cardano-sl v1.1.0! Finding such problems is the purpose of the testnet, however, and the team is ultimately pleased to have found such a problem before it made it to mainnet.

NETWORKING

This week the networking team has been working towards the 1.0.0 release and making progress with peer discovery. The error policies pull request was merged, the typed-protocol driver was refactored to accommodate timeouts and size limits, and the logging messages were reviewed. Progress was also made on the peer-to-peer governor, as well as a basic simulation with dummy cold peer selection, which allows the team to draw diagrams of the network. Work is also being done to refactor the ntp-client, which was imported from cardano-sl.

DEVOPS

The DevOps team has been working with the Daedalus team this week to support the release of a new version of the balance check Daedalus client.

CARDANO DECENTRALIZATION

This week the team finished up some work on the invalid trace generation at the level of transactions on the ledger layer. This provided a nice point to stop, review how the work on invalid traces was going, and decide how to proceed. In particular, the team has decided to go with QuickCheck instead of Hedgehog for testing purposes, and to stop work (for now) on invalid traces. The invalid traces are very helpful for conformance testing, but the plan for the Shelley ledger remains to continue using the exec specification as long as possible, which means no conformance testing. The team can now focus on finishing all the generators.

Elsewhere, the team now has the Shelley ledger API implemented, as needed by the consensus layer. The team also fixed several small issues that came from the auditors this week.

GOGUEN

This week the Plutus team started work on updating the multicurrency paper, as well as investigating how to handle mutually recursive datatypes. In addition, the team worked on some minor PureScript build improvements.

The Marlowe team added an extra restriction to MarloweÅfs symbolic counterexamples so that slot numbers in transaction slot intervals cannot be negative. They also changed the order of tabs in the Marlowe Playground to be more intuitive.

WEEKLY DEVELOPMENT REPORT

Daedalus

Wallet

This week the team has been entirely focused on delivery of the balance check version of Daedalus for the Incentivized Testnet. All the required UI and UX changes have been implemented, reviewed, and tested.

App Platform

The application platform team has been supporting the main Daedalus team in their efforts for the Incentivized Testnet this week, so there are no updates.

Cardano Explorer

This week the team completed all the work needed for the first release of the Cardano Explorer, which covers all the Byron-era features. The code has been reviewed, and the application is now in the QA phase.

Wallet backend

Wallet

This week the team has been working on delegation support in the wallet while also juggling Jörmungandr release candidates. The team is making an effort to stay in sync with Jörmungandr latest to avoid having to resolve any last-minute integration issues, as well as to allow other IOHK engineers to play with it on staging environments.

Networking

This week the networking team tracked down a bug in the ledger, as well as making progress with profiling the peer-to-peer graph simulation to be able to simulate larger graphs. Additional work is also being done on the peer-to-peer governor and lazy multiplexer, which can start and stop mini-protocols.

The team revisited the Praos paper calculations of the probability of diffusing a block within a slot time this week. They also merged the Windows named pipes API patch, and reviewed and merged a long-standing patch that implemented the orElse combinator of MonadSTM. This includes a formalization of the operational semantics of STM, as described in the paper A Compositional Theory for STM Haskell, which will allow the team to check that the implementation of STM and the GHC implementation agree with the formal operational semantics.

DevOps

The DevOps team has been working with the Daedalus team this week to support the release of a balance check Daedalus client for the early phases of the Incentivized Testnet.

Cardano Decentralization

The team spent a lot of time this week addressing property testing shrinkage issues, as well as doing other work on invalid trace generation. The decision was also made this week to remove from the implementation the formal specification requirement that stake keys need to witness their registration certificates. Making this simplification has allowed the team to simplify the implementation of several other related components.

Goguen

This week the Plutus team did a significant amount of work on ledger updates, refactoring a variety of things relating to addresses and hashes. The main improvements from this work included deleting outdated functionality, hiding the constructor of address (to ensure that addresses can only be constructed in prescribed ways), updating TxId and Address so that they can be used on-chain, and consequently removing TxHash.

In addition, the team started to work on a document that describes the architecture of the Plutus repository as well as some general performance improvements.

The Marlowe team worked on a review of styling and user experience improvements within the Marlowe Playground. Work has also started on updating the Marlowe testnet to reflect recent functional changes.

WEEKLY DEVELOPMENT REPORT

Daedalus

Wallet

This week the team continued with work on UI changes required for the Incentivized Testnet Daedalus release. These included enabling parallel wallet restorations, updating the wallet restoration and syncing notifications, and implementing legacy wallet UI changes. This week also saw the addition of a network information overlay which will contain all relevant network information for users. The bug icon has been replaced with a network badge which, when clicked, opens the network information overlay. The rewards screen has also been updated. ​

App Platform

The team was focused on helping out with work on Daedalus and the new Cardano Explorer this week, so there are no updates for the application platform. ​

Cardano Explorer

This week the team worked on solidifying the client-side state handling, formalizing the search and data feeds features, and implementing integration tests.

A continuous deployment process was established for the application bundle this week, utilizing environment configuration to build a range of targets covering different networks and Cardano eras. This process is intended to allow iterative releases at various stages of development with a short loop to QA and production.

Wallet backend

This week the team has been working on major refactoring of the wallet core engine, which will enable the handling of new Shelley addresses with delegation capability. The team is also actively working on delegation certificate submission and recognition to allow users to take part in delegation.

Meanwhile, work has progressed on stake distribution, despite some limitations in the Jörmungandr REST API. An ‘apparent performance’ metric is now provided for each stake pool, allowing them to be ordered by a frontend application.

Finally, a few minor bug fixes were implemented this week.

Networking

This week the team has been working on network-mux library changes required by the peer-to-peer component, specifically delayed or on-demand startup of mini-protocols and mini-protocol shutdown and restart. Progress has been made on the peer-to-peer component itself as well, and the team has been working on graph simulations of a peer-to-peer network using the following topologies: cycle, cycle with a fixed number of shortcuts (as in Poldercast), and random regular graphs (graphs with a fixed number of outgoing edges). The results show that random regular graphs exhibit expected block diffusion results that are better, both in absolute value and scaling behavior, than the use of cycles or cycles with shortcuts. This is consistent with the complexity results of small-world graphs.

Elsewhere, the team submitted the first pull request with bindings to Windows asynchronous I/O operatics (using I/O completion ports). The NTP client was also extracted from the cardano-sl repository and is currently under code review.

DevOps

The DevOps team has been busy this week supporting the work for the Incentivized Testnet.

Cardano Decentralization

This week the team worked on implementing an infrastructure for measuring the actual heap space used by data structures, and for checking that they do not contain any unexpected thunks. It is now possible to validate, as part of continuous integration processes, that there are no space leaks.

Work has also been done to reduce the memory usage of the ledger state and the ledger database in the consensus layer. Thanks to the use of purely functional data structures, it’s possible for the ledger database to share data across multiple historical snapshots of the ledger, which are kept to support rollback and the validation of potential network forks. As a result, the ledger database – which keeps 22 snapshots of the ledger state in memory at all times – now uses only 5% more memory than a single copy of the ledger state.

Goguen

This week the Plutus team integrated their published papers into the top-level README file to increase their visibility. They also provided some type checking for input-output pairs.

The team also updated the types used in the ledger contracts for the debug output that is printed. They also made some general improvements to Nix, purty, and asciidoctor material across the project.

The Marlowe team worked on the Marlowe 3 interpreter implementation this week.

WEEKLY DEVELOPMENT REPORT

DAEDALUS

Cardano Wallet

This week the team implemented a new theme for the Incentivized Testnet Daedalus version, which will make it easy for users to distinguish between mainnet and testnet versions of Daedalus at a glance. Work also progressed on the UI changes required for the same release, which include:

• Enabling restoration of wallets with both 12-word and 15-word recovery phrases
• Making a spending password mandatory
• Implementing address validation changes
• Updating the 'Rewards' screen
• Removing 'Decentralization countdown', 'Decentralization info' and 'Staking epochs' screens

Finally, the newsfeed feature was extended by adding support for opening in-app screens or triggering internal actions such as downloading the logs from the newsfeed items.

Application Platform

The team was focused on helping out with work on the new Cardano Explorer this week, so there are no updates for the application platform.

Cardano Explorer

This week the team upgraded the schema and query validation within Cardano GraphQL to remove specific validation checks, and instead use Hasura to enforce row limits to protect the PostgreSQL database from abuse. The backend services were also load tested and compared with the benchmark and performance improvements listed for future optimization. Application caching was also added, driven by the slot duration of the network.

The explorer now has the concept of a Cardano era and network type, making it possible to target versions for different use-cases. Client-side application state and data fetching are currently being implemented.

WALLET BACKEND

Wallet

This week the team focused on cleaning up and preparing for the upcoming release. Work is progressing on cross-compiling the wallet backend to different platforms, and both Linux and OSX are now supported. Windows cross-compilation is still underway and should be completed soon.

Meanwhile, the team has also worked on a new coin selection algorithm to allow efficiently moving coins from a wallet to another (in particular, from a Byron wallet) while keeping an adequate UTXO distribution and taking care of possible dust coins.

NETWORKING

This week the team has been progressing with the design of the peer-to-peer selection mechanism. A Jupyter Notebook was also configured, which is used for peer-to-peer graph simulations. 

Elsewhere, the team updated the Byron proxy so that it can handle new transactions. The network magic number is now read from genesis configuration, and this change has been implemented across various components (the Byron proxy, Cardano node, and Cardano explorer). 

Finally, work continues on asynchronous I/O for Windows (using completion ports).

DEVOPS

There are no updates this week.

CARDANO DECENTRALIZATION

This week the transaction generator was extended to generate transactions with a payload, which simulates Plutus script submissions to the blockchain. The generator can also accept multiple inputs for generating transactions now. Moreover, the new node now reports 'chain density' as a quality metric for the last k blocks. New metrics are displayed in the LiveView of the node and exported via EKG to Prometheus for further monitoring.

Testing of the launcher and update mechanism with QA resumed this week.

GOGUEN

This week the Plutus team made updates to contracts to generalize the error type and allow users to choose what they want to use there. Minor bug fixes were also made to the Resumable contract and the JSON instance.

The team also started work on an unexpected interpreter error in the Plutus Playground. In addition, they updated cardano-crypto to allow users to install things that have dependencies on it, such as plutus-exe.

Updates were made to handleBlockchainEvents to run repeatedly so that when an endpoint performs multiple actions it can handle all requests together, rather than a single request.

The Marlowe team made some visual and layout refinements to the Marlowe Playground and also applied some OS X fixes.