Status Updates

Weekly Development Report

DAEDALUS

Wallet

This week the team finished all the work planned for the upcoming Daedalus 0.12.0 release. All of the code changes have been reviewed, approved, and tested. The team has prepared a release branch which will go through the QA process.

In order to improve the test coverage of the IPC-driven Cardano node / Daedalus communication, the team has worked closely with DevOps on the development of Cardano node fault injection features which will enable the team to predictably test Cardano node behavior.

The team is still working on the optimization of the application environment setup and disabling of the Node JS integration in the rendering process. Both of these tasks introduce a lot of code changes, which makes the code review process more demanding and time-consuming.

As a part of the regular maintenance tasks, the team is working on Flow.js setup improvements. The team has fixed a small issue with the Daedalus antivirus notification close button's background color during hover state, and another issue with maximum decimal places handling within the React-Polymorph's NumericInput component.

App Platform

This week the team collaborated with DevOps to finalize the Cardano SL Docker image improvements, support the current platform prototype, and provide a development environment independent of Nix. In addition to single nodes, a new demo cluster image will be available as an alternative to the Nix script.

The Cardano SL service was extracted into a separate module that now serves its own GraphQL endpoint, which the platform can use with first-class support. This makes the service more flexible, allowing it to operate on its own or be used in other projects. A Dockerfile based on Alpine was included for container deployments.

A performance and usability study of the new API was initiated using a fork of Daedalus, which serves as a good test bed for side-by-side comparisons.

WALLET BACKEND

Since the team has completed all of the main features for the wallet backend, the engineers spent the week on addressing bugs found during integration of unreleased code.

Additionally, the engineers completed working on improving the performance of pre-filtering of API responses.

NETWORKING

Last week the team held a meeting with several senior engineers to discuss the integration of the Communication Protocol Design. The team architects are in the process of using a chain selection implementation design, created by a senior engineer, to power the protocol by way of their typed-transition approach.

DEVOPS

DevOps updated the mainnet cluster to run Cardano 1.3.1 and published the Daedalus 0.11.1 update to the blockchain. An engineer also started discussions with a developer working on the log-classifier project to arrange a deployment. Once deployed it will keep Technical Support and the developers current on the most common problems found in logs provided by bug reports.

An engineer on the team discovered and diagnosed an infinite loop in Cardano wallet, which was soon fixed by developers. Issues were discovered in Linux Daedalus development builds, including hyperlinks failing to open in the browser and icons failing to render. An engineer implemented fixes for both. The hyperlink issue was due to sandboxing provided by nix-bundle not giving Daedalus an obvious way to load the browser. To work around this the engineer implemented a patch which allows nix-bundle users to enable an escape hatch mechanism to the host environment. More work was done to add failure injection to cardano-node to facilitate Daedalus testing.

The team continues to improve continuous integration, continuous deployment, and the developer experience. An engineer made progress separating the build and check phases in nix builds, so we can optimize scheduling of large evaluations on Hydra continuous integration. As more modular Cardano repositories are being created, DevOps engineers have been engaging with developers to ensure that testing and some form of continuous integration are being implemented. Plans have also been made to implement a small proof-of-concept Bazel+nix solution in one of the existing repositories, so engineers throughout the company can evaluate whether Bazel may be a helpful addition to their codebases. To simplify integrating, testing, and using Cardano node, wallet, and explorer, a DevOps engineer has started working on continuous deployment for publishing Docker images to Docker Hub

CARDANO DECENTRALIZATION

Design

A senior architect has made further revisions to the delegation design specification since last week and the latest spec has been sent out for review.

A senior engineer indicated that they need more time to work on the technical implementation plan and will look to have this complete with high-level estimates shortly. This will then be circulated to the development team for independent review and feedback.

Development

The team worked to find and fix a performance regression, as well as write documentation. A team member spent their week working with the sl-formal-spec team and will continue the work this week.

Furthermore, the team’s architect has prepared an implementation of Ouroboros BFT in the existing code.

GOGUEN

The team worked to develop a draft test and quality plan for Plutus and Marlowe, as well as enhancing the test and quality plan for K/IELE to include gas model testing and performance and benchmarking.

Also, the team oversaw RuntimeVerification work on the Plutus-IELE compiler, LLVM backend, K-Haskell, gas Model, and logging and extending the Remix IDE for KEVM.

Team members concentrated on Plutus and Marlowe development, including DevOps work on mockchain and GHCJS which resulted in a successful execution of the first Plutus program on mockchain.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Performance Management Specialist, Web Tester, Director of Product Management, Rust Software Engineer, Haskell Trainer, Technical Writer as well as several others. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

Last week the team extended the status messages shown on the "Loading" screen. Along with the existing "Connecting" and "Syncing" messages, users will now also see "Starting Cardano Node", "Stopping Cardano Node", and "Updating Cardano Node" messages which was made possible by the introduction of IPC driven Cardano node / Daedalus communication.

As a part of the regular maintenance tasks, the team is working on the optimization of the application environment setup and disabling of the Node JS integration in the rendering process.

App Platform

The team performed R&D on a suitable base Docker image for front-end only applications. Volume provisioning was upgraded to use data streams, avoiding temporary filesystem access, and container limit resource control measures investigated. A specification was produced to work with the backend team for developing a configurable Cardano-SL Docker image to improve developer experience in addition to running any type of node in a container-based environment. The Swagger to GraphQL adaptor was also improved to read the spec file dynamically from the docs endpoint, and additional filters added to align with the strict GraphQL character specification.

WALLET BACKEND

Last week the team concentrated on code optimization as well as addressing tickets related to improving the user experience and the behavior of the transaction history.

NETWORKING

There is no update this week.

DEVOPS

DevOps deployed updates last week, including the Cardano 1.3.1 RC2 proposal to staging for testing, updates to the K-EVM and IELE testnets, and supporting a developer deploy log-classifier to a server.

To facilitate Daedalus front-end testing and development, a DevOps engineer started adding failure injection features to Cardano-SL. Another engineer helped diagnose and fix a bug in Cardano-SL which caused it to hang during shutdown in certain scenarios.

The DevOps team continued to improve build and CI tooling. An engineer has been improving their nix-tools repository to facilitate Cardano-SL cross-compilation in a way that is compatible with existing nix-based workflows. Build and check (aka "test") phases in large nix dependency graphs such as Cardano-SL are interleaved inefficiently, so two engineers have been working on separating the build and check phases more. Another engineer improved nix usage documentation in Cardano-SL and implemented cardano.project.freeze file generation to improve the caching experience during development. Initial build, CI, and infrastructure discussions have started for a new Cardano repository. The team has also started working with a developer who is using macOS Mojave to get nix builds working.

CARDANO DECENTRALIZATION

The team is already translating the specification into an executable specification/prototype. There are also some modifications to the design document, both to clarify some aspects with feedback from the prototyping team, and to hone some last rough edges. The Daedalus team is reviewing updated Delegation spec to implement wallet requirements.

A senior engineer is continuing work on the Technical Implementation Plan. This implementation plan will provide the tasks, milestones, dependencies, success criteria and estimates. This will be reviewed by the developers to ensure alignment.

GOGUEN

There is no update this week.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Performance Management Specialist, Web Tester, Director of Product Management, Rust Software Engineer, Haskell Trainer, Technical Writer as well as several others. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

Last week the team finished testing of the IPC driven Cardano node / Daedalus communication implementation. Smaller issues with the defensive Cardano node startup setup have been identified and fixed. The team added additional logging which should help to debug any potential future issues.

The team has implemented a lock-file solution which prevents multiple Daedalus instances from running against the same network (Mainnet, Testnet or Staging). A prerequisite for this was a complete separation of Daedalus state directory based on the network Daedalus is connecting to which was also implemented last week.

In the scope of the V1 API integration, the team introduced more fine-grained error messages for the transaction fee calculation V1 API endpoint. Error messages now cover all three potential cases:

1. Transaction amount plus the transaction fee exceed the wallet's balance,

2. Transaction amount exceeds the wallet's balance,

3. Having pending transactions which include wallet's addresses which contain funds needed for the new transaction.

As a part of the regular maintenance tasks, the team has introduced a helpful environment variable which is used to prevent Daedalus from closing at the end of the acceptance-tests run. This makes debugging test failures much easier since leaving Daedalus open preserves the last known state. In order to further improve the acceptance tests, the team implemented automatic recording of failed test cases screen states. The team has also removed the last parts of the ETC code from the codebase since Daedalus with ETC support is now being developed in a separate GitHub repository as a separate project.

App Platform

The team built out the GraphQL API endpoint to require certificate-based authentication and support for consuming remote schemas published by services or applications. A client certificate is issued and written into the application's mounted volume on startup. The Cardano SL service Swagger adaptor was also enhanced to enable TLS authentication with the REST API.

WALLET BACKEND

Last week the team focused on improving the memory footprint of the acid-state database for the new wallet. After a set of tests on unreleased code, the team noticed that syncing the backend from start would significantly increase the database size on disk, which lead the team to putting their efforts into improving acid-state disk space for the new data layer and successfully fixed the bug.

Also, the team focused on investigating if backfilling (i.e. what happens when a wallet is restarted and needs to apply the blocks) was working correctly, and found some discrepancies. These discrepancies were then addressed and fixed.

NETWORKING

The team continued their work on Communication Protocol Design. They now have new static relay and core nodes. Also, the team created simple simulations, investigation priority list and much better tests for the simulated environment. The team now has a nix build tool that will run on Hydra (Continuous Integration Server). This week the team are working on node simulations and their properties and more test, and will further improve documentation and description of the protocol.

DEVOPS

A DevOps engineer prepared and announced the Daedalus RC2 builds for Cardano-SL 1.3.1. Later DevOps engineers updated the internal staging cluster nodes to that same revision of Cardano-SL.

Early last week, nodes on both the internal staging and public mainnet Cardano-SL clusters stopped responding and transmitting metrics. Initial investigations suggested one of the cloud provider's region was a factor, as most of the affected nodes were based there. The cloud provider acknowledged degraded network connectivity, and it was soon fixed. Nodes in that region and nodes communicating with them regained health automatically as expected.

Two DevOps engineers gave internal technical presentations. One demonstrated their evaluation of Bors, a PR and CI workflow automation tool, and discussed a strategy for continuing to evaluate and incrementally adopt it in more IOHK GitHub repositories, starting with the repository maintained exclusively by DevOps. The other engineer presented on Nix and common development and testing use cases within IOHK.

The team continued addressing pain points with CI responsiveness. A large spike in build times was detected and traced to Nix expressions which inadvertently were building all of nixpkgs, including non-dependencies. DevOps engineers met with developers working on the Snack build tool which aims to support incremental nix builds for Haskell projects. They discussed insights from our experimental evaluations, and what improvements would be necessary for using it more widely at IOHK. An engineer made progress on an incremental stack-based build strategy for CI builds, which is expected to help in PRs with multiple commits. Future improvements will be facilitated by refactorings and fixes, like changes to CI and deployment configurations that reflect repository restructuring, and extracting nix overlays from an increasingly complicated default.nix for Cardano-SL into separate files.

CARDANO DECENTRALIZATION

A detailed work breakdown structure of the delegation Technical Implementation Plan is in progress and will be finalized by the end of this week.

GOGUEN

There is no update this week.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Performance Management Specialist, Web Tester, Director of Product Management, Rust Software Engineer, Haskell Trainer, Technical Writer as well as several others. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

Last week, the team finished the implementation of the new Cardano wallet data layer migration screen and continued to work on testing of the IPC driven Cardano node / Daedalus communication.

In the scope of the V1 API integration, the team introduced a new error message which is shown in case the user tries to create or restore a wallet by using an example backup mnemonic phrase from the Cardano documentation.

As a part of the regular maintenance tasks, the team is working on several tasks such as the optimization of the application environment setup and disabling of the Node JS integration in the rendering process.

App Platform

The team designed the Cardano SL service using an adaptor to generate a GraphQL schema from the published Swagger spec. The platform hosts each service schema using a namespaced hierarchy, which can be explored using the included GraphQL Playground. The platform is responsible for managing certificate-based authentication with applications and other services and enforcing access control based on service definitions and user permissions.

WALLET BACKEND

Since all of the main features have been successfully completed for upcoming 1.3.1. release, the engineers are now concentrating on addressing bugs found during integration in an unreleased version of the code.

Additionally, the team focused on integration testing. They found and solved the issue of wallet restoration being stuck at 100%. While testing out the wallet, the team noticed how the acid-state database of a restored wallet occupied almost 1GB, and went on to find out how to reduce the number of things that are stored in the DB. This originated into a pull request that has been opened which mitigates the issue. More work in this area will be done this week.

NETWORKING

A senior engineer has added a new (simpler) chain producer state with types and property tests and a new chain implementation for Communication Protocol Design. Another team member have updated the protocol with the new chain and consumer protocol changes, extending them where required. They also finished merging senior engineer's changes as well as their own generators for property testing.

DEVOPS

DevOps continued optimizing CI response times and reduce developer toil. Hydra's Linux build agents, which were previously virtual machines, have been replaced by bare metal instances to speed up CI times. Some engineers are experimenting with the snack build tool on cardano-sl to get finer grain caching and incremental builds using Nix. A major blocker to using snack is evaluations before any of the builds begin are time-consuming. We improved evaluation time some, but not enough yet to be practical. To improve the developer experience an engineer added explicit support for cabal new-build under a nix-shell with binary cache reuse. Support for centralized binary caching for the nix-shell environment was also added. Internal evaluation of bors-ng to improve the PR approval and merging workflow continues. Periodically macOS build agents have been running out of disk space, so monitoring was improved and tested.

As developers continued work on the required address discrimination feature for the Testnet, DevOps helped to test and isolate integration bugs. The Testnet configuration and cluster have also been updated to reflect the latest state. Sources of non-determinism were identified and removed in Cardano's ./pkgs/generate.sh to ensure developers and CI get the same result. A NixOps deployment issue was traced to case-insensitivity in hostnames; an issue and PR were filed upstream. This helped integration work for Plutus and Cardano progress. Wallet integration tests sometimes exhibit false positive results, and an engineer has started investigating. Cardano error logs were too generic when a worker program crashed, so a DevOps engineer submitted a PR to capture more context.

CARDANO DECENTRALIZATION

The team have been working on the detailed technical implementation plan where timelines and allocation of resources are being defined.

A Test and Quality plan has been drafted where a number of internal teams have collaborated to provide their input and feedback. This document will be a living document over the next few weeks until the Networking design components are completed in order for us to go to the next level of detail.

GOGUEN

There is no update this week.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us as a Performance Management Specialist, Web Tester, Director of Product Management, Rust Software Engineer, Haskell Trainer, Technical Writer, Senior Haskell Developer - Prototyping as well as several others. Please see the IOHK Careers page for more details.