Status Updates

Weekly Development Report

DAEDALUS

Wallet

This week the team continued working closely with the QA team on resolving newly identified issues as part of the Daedalus 0.12.0 with Cardano 2.0.0 release process. All identified issues have been successfully resolved and the fixes have been confirmed by the QA team.

The fixes include:

• a fix for a problem when starting Daedalus while the time on the user's machine is incorrect with the offset to the past

• a fix for a problem with wrong transaction status and display of pending transaction amounts on wallet "Summary" screen during wallet restoration process

• a fix for a misconfiguration which caused debug-level logging messages to be recorded in Daedalus log files

The team is also implementing better handling for the low disk space warning. Once implemented, Daedalus will shutdown Cardano node if disk space gets too low, and show an error screen asking the user to free up more disk space to continue using Daedalus.

In the scope of regular maintenance tasks, the team has continued to work on quarterly Daedalus dependencies updates and started refactoring layout components, which should reduce re-renderings and increase performance. The team has also fixed an issue with the acceptance tests set-up which caused false positive test failures with the application-shutdown scenarios.

App Platform

This week the team implemented the first Cardano GraphQL subscription to push changes to the Cardano node connection status over a websocket managed by Apollo Server. Apps can use this information in a number of ways, for example: to avoid or delay sending requests, disable UI elements, or present a notification on screen.

The team made Daedalus components work on Cardano wallet app by changing the build system from "create-react-app" to "nwb", which allows custom webpack customizations. This has enabled the team to use CSS submodules and SVGs the way they are used in Daedalus without breaking the build.

WALLET BACKEND

As part of the work to implement the node management and monitoring API, the team has moved most of the implementation logic related to the API from the wallet to the node. The wallet will be able to reuse the logic by using cardano-sl as a library, which will improve code reuse between cardano-sl and cardano-wallet repositories.

The team also concentrated on moving and renaming the BIP39 module to cardano-sl as mnemonic. The team have reversed this dependency and have made the BIP-0039 implementation part of cardano-sl as a new mnemonic package. In addition, the engineers have relocated a few executables and scripts inside this new mnemonic package.

NETWORKING

The team has started documenting their initial ideas for the new approach to the peer discovery layer. The next steps are to finish writing it up, propose an algorithm, analyze it from a threat perspective, and start building an initial simulation

A senior engineer has also started working on block download logic, with the next step being to sync previous work on block download protocol with their ideas.

DEVOPS

This week the DevOps team were operating at a reduced capacity, and were largely busy with incident management, but also continued the usual stream of CI improvements and maintenance work.

CI efforts included improving cross compilation ahead of final production readiness, migration of our build pipeline definition to the new nix-tool mechanism, and continued CSL submodularization in support of the Cardano rewrite.Daedalus saw some issue triage, and a new EKG dashboard feature.The team also worked on improving support desk automation to ease workflow, and reviewed some of the plans for Shelley.

CARDANO DECENTRALIZATION

Work on the initial architecture continues, and the team switched to a new version for the architecture for cardano-shell. The team spent time merging the initial architecture and working with the monitoring team on integrating the code. Two of the team members are working on a logging prototype to be integrated with the node-shell and the networking efforts in ouroboros-network.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Rust Software Engineer, Haskell Trainer as well as several others. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team fixed all newly discovered issues during the first QA phase of the upcoming Daedalus 0.12.0 and Cardano 2.0.0 release.

The fixes and improvements include:

• adding the network name (mainnet or testnet) to the Daedalus title bar so that it is easier to identify which network Daedalus is running on

• removal of wallet import and export features from Daedalus testnet build

• a fix for a caret positioning issue on the transaction amount input field

• a fix for a low-quality Daedalus window icon on Linux

• a fix for the support-request feature submission issue where it was impossible to submit a new support request if the user had already downloaded the logs prior to opening the support-request dialog

The team has improved acceptance test performance by using Cardano node state caching to reduce screen reload time. The full test suite now takes approximately 9 minutes to complete, a reduction of 5 minutes.

In the scope of regular maintenance tasks, the team has continued to work on quarterly Daedalus dependencies updates.

A new version of React-Polymorph framework has been released on npm. It contains several important changes such as refactored flow-type setup and improved flow-type declarations and lint fixes which are targeted into making the framework more stable and robust.

App Platform

This week the team started to migrate existing Daedalus components into the Cardano wallet app running on the Daedalus app platform. The core idea was to reuse as much of Daedalus' functionality as possible while using the new GraphQL layer on top of Cardano SL to perform blockchain operations.

WALLET BACKEND

Last week the team set up a new cardano-wallet GitHub repo with a complete CI pipeline which includes builds, tests with coverage reports, linters, and automated export of the API documentation. The engineers also completed analysis on the submission layer and the node-to-node protocol, and produced outputs with detailed diagrams and explanations about the current behaviour. It appears that not much work will be needed to decouple at the submission level, so the team started removing cardano-sl dependencies on cardano--wallet to achieve full structural decoupling.. The team also moved the node monitoring and node management API onto core nodes, and adjusted the node’s command line interface accordingly.

NETWORKING

A senior engineer worked on generalization of the typed protocol definitions so that they permit early yielding/awaiting for latency hiding. Other team members analyzed requirements for the peer discover layer and are looking for an in-house solution.

DEVOPS

This week DevOps were mostly busy with extending the scope of CI and incident management. Additionally, the team worked on testing automation, integration of new repositories, Zendesk support, and the Goguen project.

The CI side saw time spent on multiple fronts. Efficiency work included continued effort on Windows cross compilation (in particular the massive update of the Cardano software stack to GHC 8.4/LTS 12), merge queue workflow automation, a CI prototype workstream, and improved machine support.

The CI scope expansion was due to the continued enrollment of repositories carrying the Cardano rewrite, further work on cardano-shell, cardano-wallet, and the Plutus project, and also deeper automation of cluster testing.

Goguen clusters also suffered from a number of operational issues and had to receive a lot of our attention - but we also managed to refactor their deployment toolstack, clean up the DNS infrastructure, improve documentation, and fix bugs to create a smoother engineer workflow.

Last but not least, we also helped with support desk debugger infrastructure and assisted in preparation of the 2.0.0 release.

CARDANO DECENTRALIZATION

The team has begun an initial proof of concept for the integration of a subset of the executable specification of the ledger rules with the node prototype, and a review of the interactions and interface. A team member presented the architecture of the node and the role of the consensus layer to the company, and has made further progress on the design which now includes chain validation.

Most of the ledger interface is defined, although this can only be fully completed after the ledger layer team is done. Engineers settled on a basic design for on-disk representation of the immutable tail of the blockchain and interaction with the volatile prefix. The team has started mocking out a test implementation, and also started work on a test infrastructure that will allow them to test things like disk failure.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Rust Software Engineer, Haskell Trainer as well as several others. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team fixed all the issues discovered during first QA phase of the upcoming Daedalus 0.12.0 and Cardano 2.0.0 release.

The fixes include:

• a fix for a problem with maximizing the Daedalus window on Linux, where the application menu would be hidden in the maximized state

• a fix for an issue with orphaned Daedalus processes after trying to launch multiple Daedalus instances on Windows

• a fix for launcher misconfiguration which caused the wrong error message to be displayed when a second Daedalus instance was launched

The team has removed the "Antivirus software performance impact" notification shown on Windows during wallet restoration. The performance issues have been resolved in the upcoming Cardano 2.0.0 release.

An error message for "Not enough disk space" has been implemented, reviewed, and tested. This message is shown during Daedalus startup if there is not enough disk space available on the user's machine for Daedalus to run successfully.

In the scope of regular maintenance tasks, the team has cleaned up the Daedalus theme directory and fixed a failing Cardano Node "Apply Node update" acceptance test. The team has also started to work on quarterly Daedalus dependencies updates. This task will continue throughout the upcoming week, since the team has more than 140 dependencies to update.

App Platform

This week the team continued with the access control layer, making changes to the UI to present granular service operations at the point of application install, which is used to control API access for the defined service. A couple of configuration issues with the Docker-based Cardano demo cluster were identified, reported to DevOps, and new versions tested. A strategy for building out the platform-based wallet app past a prototype was determined, and work started to use existing components from the Daedalus codebase.

WALLET BACKEND

The team have confirmed the hard fork mechanism (HFM) to work on a devnet cluster. The engineers have cleaned up the code so it’s suitable for production, and are rerunning tests to double-check the results.

NETWORKING

Last week the team settled on the comms protocol design, although there might be some alterations in the future. Currently, the team is focusing on implementation.

The engineers have the protocol in Sim, but are currently switching to a framework which uses a type system to ensure the client and server sides of the protocol match.

DEVOPS

The team continued nix-tools integration work for building the cardano-shell repo.

Engineers added some tooling to simplify PR review, build, and merge workflow for the cardano-chain repo. Automated build and test proof of concept work continued in the Plutus repo, and the team also integrated an improved CI build and test scheduler for Plutus. During testing, the team experienced more Windows CI build issues, including timeouts and MAX_PATH limit, which were later addressed.

The engineers also finished an initial proof of concept fallback strategy using Windows servers, and resolved GitHub rate limit breaches by reducing a regular status check footprint.

Furthermore, the team deployed additional utility for the Support Portal, and started work on a backend for nixops. Team members updated Hydra server to support jobsets involving git submodules.

In terms of operational maintenance, the team fixed websockets on Cardano testnet explorer, and provided operational support for IELE and Mantis testnets.

Also, the team fixed a port collision bug when running multiple editions of Daedalus for different clusters, and cleaned up stale benchmarking logs from a shared deployer machine.

CARDANO DECENTRALIZATION

Design

The first prototype of the consensus layer was written in a process calculus style; this prototype can be found in the ouroboros-prototype GitHub repo. The team designed it to match as closely as possible the style of formal specification that will ultimately be used for formal verification.

The networking team, however, implemented their algorithms in a different style. Since the consensus layer will be sitting on top of the network layer, it made sense to restructure the consensus layer to use abstractions provided by the network layer. The team’s first attempt resulted in a coupling between the two layers that was a little too close, however. The initial attempt was nonetheless useful to understand the issues at hand (and they spotted a bug in the network protocol in the process too).

As of the end of last week, it was decided to decouple the two layers further, introducing some abstractions into the network layer that will allow the network layer and the consensus layer to be developed independently, and yet use the network layer in the consensus layer. This redesign is not fully complete yet, but we're close.

Meanwhile, a team member is working on some integration with a mock ledger layer. Although the ledger sits on top of the consensus layer, the consensus layer needs to be aware of the ledger layer since some decisions at the consensus layer depend on the state of the ledger. A mock ledger layer allows us to implement such decisions without being blocked on the work done by the ledger layer team. It works on the back of the UTxO model that has been previously developed for the wallet.

Finally, a senior researcher is working on designing some concrete crypto primitives that are needed in the Ouroboros implementation. These are not necessarily intended to be final, but are to explore the design space, and identify the interface to these crypto primitives, and which assumptions we can and cannot make.

The team is converging on a design in which we can easily swap in and out

• various representations of blocks

• various Ouroboros flavors (BFT, Praos, ..)

• various ledger layers (real, mock)

• various crypto implementations (real, mock)

This allows to focus on key design without pinning us down to concrete decisions that might later be changed.

Development

Last week a senior team member helped find and fix a performance regression, wrote some docs, and reviewed the new cardano-chain project. Another team member worked on the sl-formal-spec and will continue this week. Additionally, the developers are preparing for implementation of Ouroboros BFT in the existing code.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Rust Software Engineer, Haskell Trainer as well as several others. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team continued working on adding test coverage for the IPC-driven Cardano node / Daedalus communication.

The team has improved the NTP logic, which now covers an edge case where the user's machine can’t reach the NTP service and check if local time is in sync with global time. In this case, the user will be presented with an error screen explaining the problem and a link to the Support Portal page with more details about the problem and ways to fix it. The user will be allowed to continue using Daedalus without time sync checks, although this is not advised and should only be done if there is no way for the user to fix the NTP service access.

App Platform

This week the team worked on the API access control layer, preparing the system to issue a JSON web token containing the user's service authorization. An integration test suite was added and a compatibility issue with a dependent library resolved in preparation for progressing the GraphQL service from a prototype to a more developed project. The team also began research into how GraphQL subscriptions can be utilized to offer streams of data to API consumers, and two events were identified for development as part of the value-add layer over the REST API. Progress was made on the platform-based wallet app, including making transactions against the Cardano GraphQL API.

WALLET BACKEND

Wallet

The team completed the first version of a new implementation of Cardano wallet based on Cardano wallet formal specifications. It will be shipped to users with Cardano 2.0.0. after QA is completed.

Testnet

The engineers found some issues with the new wallet implementation while working on the upcoming 2.0.0 release (previously 1.4.0). When transacting, ada would be put back into the user’s account an hour later - this is now fixed and waiting for reviews to be completed.

NETWORKING

Last week a meeting was held with several key engineers and researchers regarding communication protocol design integration. The senior integration engineer is in the process of using the chain selection implementation to power the protocol by way of a typed-transition approach.

DEVOPS

This week, DevOps were busy supporting the upcoming 2.0.0 release (previously 1.4.0) and the longer-term Cardano rewrite. There was also the usual assortment of continuous integration (CI) wrangling, Daedalus, and Goguen work. Last but not least, the team grew and welcomed a new DevOps member.

With regards to CI, the team continued work on the Windows cross-compilation, which is being prepared for a final delivery after the 2.0.0 release, in addition to setting up a CI pipeline for Windows jobs. CI integration for cardano-chain, cardano-wallet, and cardano-shell is also progressing well as part of the Cardano rewrite support.

Further work went into separating build and test derivations, which will allow the team to reduce time from PR submission to approval on Github. The team also started working on automating more aspects of CI.

A major Nixpkgs update was completed, and the team worked to support the development process with version bumps and release notes for 2.0.0. A large chunk of effort went into debugging unexpected issues with address discrimination on the staging cluster. For Daedalus, the team focused on wrapping up the fault injection-based testing work.

Engineers also touched upon workflows by participating in the Cardano versioning discussion and by planning for a separate, engineer-oriented CI, which will allow team members to take full ownership of testing, and reduce disruptions associated with full production-level CI.

Goguen received an update to KEVM and IELE clusters, which unfortunately had to be promptly rolled back because of an acute operation issue discovered, which the team is helping engineers to narrow down. Finally, engineers spent time improving Goguen deployment automation, including supporting features, refactoring, reliability work and reducing divergence with the Cardano deployment stack.

CARDANO DECENTRALIZATION

Design

A senior researcher has made further revisions to the delegation design specification since last week and the latest spec has been sent out for review.

Development

Last week the team concentrated on final code optimization. Engineers helped find and fix performance regression, wrote documentation, and reviewed the new cardano-chain project.

Additionally, team members are preparing for implementation of Ouroborous BFT in the existing code.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Rust Software Engineer, Haskell Trainer as well as several others. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team finished Flow.js setup improvements. All the code is now properly flow-typed which will ease future development and improve overall stability and robustness of the application. In the scope of the QA process for the upcoming Daedalus 0.12.0 release, the team has identified Windows-specific issues with the IPC-driven Cardano node / Daedalus communication. The issues were successfully resolved and the application now runs smoothly on all of the supported platforms. The team is working on adding test coverage for the IPC-driven Cardano node / Daedalus communication based on the Cardano node fault injection features which were completed last week.

App Platform

This week the Cardano GraphQL module was moved into a separate git repository, further positioning it as a standalone service that can also be consumed as a remote schema. Improvements were made to handle TLS provisioning after startup, and a docker-compose.yml configured to orchestrate the development environment along with a Cardano demo cluster. These changes align the service with the latest approach of only holding keys in memory, after being provisioned by the Cardano edge node via IPC. Work was started on integrating access control into the GraphQL calls based on user permissions.

WALLET BACKEND

Last week the team continued testing per the release cycle and new features will be released with release 2.0. A performance issue was found in QA, but it was addressed and fixed.

NETWORKING

There is no update this week.

DEVOPS

There is no update this week.

CARDANO DECENTRALIZATION

There is no update this week.

GOGUEN

There is no update this week.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Director of Product Management, Rust Software Engineer, Haskell Trainer, Technical Writer as well as several others. Please see the IOHK Careers page for more details.