Status Updates

Weekly Development Report

DAEDALUS

Wallet

This week the team continued working on the implementation of UI designs for the Shelley delegation for stakeholders features.

The implementation of the ‘Rewards’ screen has been finalized, while work on the ‘Staking Pools’ screen is still ongoing. The team has also started with the implementation of the designs for the ‘Staking polls’ screen. This screen has the most complex UI, which will be broken down into smaller, reusable components to be used later in building other screens.

App Platform

This week the team completed initial work on the server-side JavaScript execution engine for the smart contract backend, which uses a Chromium process to isolate the untrusted contract execution, restricting the use of NodeJS to trusted operations during initialization. A security assessment was initiated in coordination with the Plutus and tools team, which will be completed along with boundary and load tests next week. A remodel of the smart contract server was also completed, strengthening the foundation before moving onto the contract off-chain state management.

BACKEND OPTIMIZATION

The team has made significant progress with testing and reliability this week. In doing so, the continuous integration setup and workflow have also been improved, allowing for better and faster feedback.

A stable implementation for the new SQLite persistent backend is almost done, and the team is looking into some deep testing techniques to improve confidence in the new implementation. This includes state-machine testing, time and space benchmarking, and fault-tolerance testing.

Meanwhile, available packages have been extended, with a deeply-tested Haskell implementation for Bech32 encoding which will power the new encoding format for addresses in Shelley. This is a positive contribution to the Haskell eco-system with some very useful features for users to prevent typing errors when entering an address by hand.

And finally, the team has also been polishing documentation and error feedback from both the API and the command-line interface.

NETWORKING

This week the team worked on improving generators and shrinkers in the property-based test suite. On the block fetch front, engineers are working on chain prioritization. The consensus layer gives us input on how to order forks, and which blocks are orchestrated for downloading.

Work was also done to merge a branch with a high-level interface for the ouroboros-network, and the team is in the process of integrating the initial handshake protocol into byron-proxy.

Additionally, a new stream of work has begun on DNS connection management, which will be important in the initial phase of the Shelley release. The team has implemented byte limits on incoming and outgoing connections. While the latter one is optional, the former limit is essential in early detection and prevention of DDoS attacks.

The team has also been improving documentation, including updating the transaction submission and handshake protocols descriptions, and working on improving CDDL tests for CBOR codecs of protocol messages.

DEVOPS

No updates this week

CARDANO DECENTRALIZATION

This week the team has been working on property testing the Byron implementation. A number of good properties to test for the ledger rules have been identified and the process of implementing them has begun. An important aspect of property testing is generating test cases: we've been building on concepts discussed in this blog by Edsko de Vries to come up with some powerful generators for UTXOs that give us good counterexamples to aid debugging. This all contributes to high-assurance code and will increase our confidence putting the rules onto mainnet.

The Windows-specific parts of the storage layer are now implemented and tested. The mempool has been implemented, although we still lack integration with the network layer to actually accept transactions. The integration with the ledger layer is cleaned up, updated for the mempool, and almost ready to merge. Apart from that, we made progress on the storage layer itself; since the storage layer is split into three parts (immutable part of the chain, that is, storing the part of the chain that will never be rolled back; volatile part of the chain, storing blocks near the tip; and storing the state of the ledger), the layer that unifies these three components requires careful thought.

GOGUEN

Updates have been made by the Plutus team to the game and vesting use cases to make them both multi-currency aware. The team also made changes to the validity range parameters and the Plutus wallet API to make things easier for script authors.

This week the Marlowe team continued working on Meadow in the cloud. The team is planning some minor edits to the Marlowe Udemy course to improve the audio in some sections, and a plan for updates to the Plutus course is underway. Positive feedback was received from internal staff on these courses in recent days.

The Education team are finalizing their plan for the remainder of 2019 and are ready to present to the executive team next week along with their Education strategy. The working group was busy with the initial chapters of the Plutus ebook.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team continued working on the implementation of UI designs for stakeholder delegation in the Daedalus wallet.

Two new screens which present information about decentralization start time and current progress have been successfully implemented, and the team is currently working on the implementation of the 'Staking Pools' screen.

In the scope of regular maintenance tasks, the team fixed a minor issue with field validation on the 'Generate address' spending password input on the 'Receive' screen, along with the minor UX issues with the renaming and deleting of a wallet on the 'Settings' screen.

App Platform

This week the team upgraded the Cardano JavaScript SDK to use the coin selection algorithm provided by the Cardano Rust project, refined the developer wallet interface, and improved high-level tests. The smart contract backend was improved to gracefully handle port collision scenarios, as well as handle startup and shutdown sequences better. The design was also determined for lifting the pub/sub-mechanics of signing a transaction created externally to a separate module so it can be used as a standalone service. Additionally, work started on a server-side JavaScript contract execution engine, which is the target for the first production-ready milestone of the smart contract stack.

BACKEND OPTIMIZATION

Lots of work completed on the integration side this week. The team has been strengthening the integration test suite in various ways and automating some previously manual bits of testing (like interacting with the command-line interface). Engineers have also started testing transactions over a local cluster of Byron nodes, with promising results.

Additionally, the team has started working on parsing the new binary format that Jörmungandr uses to represent blocks on the chain. It is now possible to decode most 'legacy' pieces of information present in a genesis block (bits related to the Shelley era have been left aside for now).

Finally, the team has been working on a new engine to support SQLite as a database option. The database schemas and low-level primitives to interact with it have been finalized and a new implementation of the database shouldn’t be far behind.

NETWORKING

The networking team has been working on a high-level interface for the network layer, simplifying the multiplexing and getting a clear interface for running network applications on top of it. The team has also integrated cardano-ledger types into the Byron proxy, and implemented a large part of the demo application which streams and validates blocks from mainnet.

Engineers have also been working on Windows support using named pipes, as well as updating the session type framework. Work has also been ongoing on ΔQ-algebra, which will be used to construct graphs of nodes and validate whether good connectivity in a peer to peer scenario is possible. The team also built a demo application for the multiplexing layer and block fetch protocol, which successfully demonstrates that full bandwidth can be used during protocol pipelining in block fetch.

DEVOPS

This week's main focus has been on integrating NixOS services for the Byron proxy. The initial work of adding NixOS services to the Byron proxy will also be adopted with command line wallets, explorer instances, and most importantly stake pool core nodes and relays. The work will also aid in providing docker containers for end users that do not want to use NixOS. These services make it easier for developers that want to proxy the current chain and run the new ledger validation against it.

The team has also been hard at work at integrating monitoring and logging tools into an internal deployment and will soon be deploying this monitoring framework to the public testnet. This framework will be provided to all stake pool operators to help them monitor their environment.

CARDANO DECENTRALIZATION

This week the team has been working on some internal demos to highlight development efforts. The first demo covers live chain validation, which shows a node pulling blocks from the network and running them through ledger validation. The second demo is of the consensus-ledger integration, which shows the permissive BFT (PBFT) consensus algorithm working with the ledger validation code. The two demos combined demonstrate that the ledger code can integrate neatly with the other core components of the Cardano node.

GOGUEN

The Plutus team worked on some refactoring and simplification of the code and reorganized some of the test cases. They were also busy updating the Plutus use cases and encoding multi-currency values. This week the Marlowe team continued working on general improvements in Meadow. Subscriptions for the online Udemy courses for both Marlowe and Plutus are now well over 5k and continue to grow. The feedback continues to be positive. The education team has been busy fine-tuning the backlog of items and held a series of prioritization sessions to form their delivery plan for the remainder of 2019. This will be presented to the executive team next week along with the education strategy. The initial chapters of the Plutus ebook are being reviewed by the team and work has started on the scenario chapters.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team started working on the implementation of the UI design for Shelley features, which will be the main focus of the team for the next couple of weeks.

In the scope of regular maintenance, the team fixed a minor issue with the caret positioning in the React-Polymorph's numeric input component.

The team has also improved the paper wallet certificate by providing its creation date and time in both the PDF document's name and the certificate's front page.

App Platform

This week the team produced the top-level developer interface for the Cardano JavaScript SDK, implemented a Cardano node provider interface to become decoupled from any one backend, and completed initial work on the wallet interface. This includes deriving the next receipt or change address for a BIP44 public account, querying the UTXO and transaction sets, and coin selection. Work was also completed on the smart contract backend to introduce in-memory contract bundle handling, and improvements to contract API server management.

BACKEND OPTIMIZATION

This week the team finalized the command-line interface for the wallet, including support for wallet management commands and an address scheme similar to BIP44 (same as Icarus/Yoroi). As a result, the wallet backend API for the corresponding commands has also been fully finalized. This was a great milestone to release a second version of the wallet backend (API and CLI) backed into a single executable for Linux.

Meanwhile, the team has also started working on two other major projects: a more production-ready database layer using SQLite; and integrating with Jörmungandr, starting with the implementation of the various new binary formats and network protocols.

NETWORKING

The networking team has been finalizing the design of the version negotiation protocol, which is an important mini-protocol for the Shelley release. It has two core functions: to provide security to the network, as well as provide a mechanism for running nodes easily when an update is scheduled and multiple versions are running across the network.

The team also implemented strict byte limits in inbound channels, and voluntary limits in outbound channels. Inbound limits are the crucial ones, since they protect the system, while outbound limits are voluntary since they can be circumvented by adversarial nodes, but are still useful to detect implementation errors. The inbound limits are static properties of each mini-protocol, which will have to be carefully designed.

DEVOPS

Continuous Integration

The team continued work on integrating nix-tools with the new repositories, including cardano-shell and plutus.

Monitoring

The team continued with monitoring tooling migration and tested the changes in an isolated environment. The changes have been finalized, but have not yet been deployed to all environments. The team will combine this roll out with the implementation of central logging changes.

Central Logging

The team finished a proof of concept for the new central logging tooling, and is currently looking at collecting the Cardano application logs via a raw JSON file. With the Shelley code base, the team will move to having iohk-monitoring-framework log directly to the new logging tool. Next steps are to add horizontal scalability to the search nodes.

Stakepool Tooling UI

The team continued designing the UI for stake pool operators to handle common tasks, and have started investigating the requirements for the backend implementation. The stake pool tooling will also support a fully command-line interface for scripting.

CARDANO DECENTRALIZATION

Research and Design

This week the team addressed a few issues in the formal and executable specifications for the Byron-Shelley transition, following review. They will be using the Shelley executable specification as the starting point for development, and all that remains is to add the final details and identify any prototype solutions that could be improved.

Development

The completion of the header-body split last week caused some conflicts with concurrent work on ledger integration, so the team spent some time this week resolving those issues. Otherwise, good progress is being made on the final component of the bespoke blockchain storage layer, and small changes have been made to the consensus layer to fully support PBFT, which will follow after OBFT.

GOGUEN

The Plutus team worked on an improvement to currency ordering within the Plutus Playgrounds as well as CSS fixes to improve the display of the blockchain table. They were also busy with updates to the multi-currency paper to reflect changes that were made recently for Plutus.

This week the Marlowe team were working on general improvements in Meadow. Subscriptions for the online Udemy courses for both Marlowe and Plutus are almost at 5000 and continue to grow. The feedback continues to be positive.

The Education team started preparing a comprehensive backlog of items from a requirements gathering exercise across the company. They will fine tune this over the next week and prioritize items to create a plan for delivery over the coming months. Work is continuing on the initial chapters of the Plutus ebook.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team worked on improving the navigation of application menu screens. Since all these screens appear in a dialog-like form, only one of them should be active at any time.

A new 'Help and Support' application menu has been introduced. The existing 'GPU Safe Mode' option has been moved from the 'Daedalus' menu into the newly added one, along with the three new items: 'Download Logs', 'Support Request' and 'Known Issues'.

The team changed the way the 'GPU Safe Mode' feature works by introducing a confirmation dialog which clarifies what this option means and when it should be used. Additionally, the UI designs for Shelley features have been completed and the team will be starting with their implementation next week.

App Platform

This week the team completed initial work on a JavaScript SDK for Cardano, which provides an interface for developers to add key management and transaction handling features to an app. The module uses the Cardano WASM bindings for cryptographic operations and uses a key management interface to allow wallet implementations to be written as an extension. An in-memory manager was implemented as the default, but support for popular hardware wallets is coming soon.

BACKEND OPTIMIZATION

With most of the team absent this week, it was an excellent opportunity to perform some deep repository restructuring and re-organization of the source code to prepare for the upcoming integration with the Rust node. After finalizing various bits of testing related to the new transaction builder, the wallet core code has been completely decoupled from any protocol-specific elements (like network serialization formats, or address representation). The core code of the wallet backend can now work in practice with any UTXO-based cryptocurrency, regardless of how the protocol works internally. This is especially powerful in terms of integrating with both the Shelley Haskell and Rust nodes. It will allow the wallet to cope with any possible discrepancies while still preserving the old backend as a viable option during the transition (and for comparison).

NETWORKING

This week, the team completed work on concurrency and state management between the threads involved in block fetch logic. Property-based tests have been included, although coverage is still partial at the moment. Engineers have started to work on the dynamic mux layer and are also running an experiment on using named pipes on windows for connecting with clients. The team has also identified and fixed some dependency problems in the Byron proxy related to cardano-sl and presented a demo of the Byron proxy. Documentation of Shelley connectivity management has begun, as well as designing behavioral objectives.

DEVOPS

Continuous Integration

The team continued work on integrating nix-tools with the new repositories, including cardano-shell and plutus.

Monitoring

The team continued with monitoring tooling migration and tested the changes in an isolated environment. The changes have been finalized, but have not yet been deployed to all environments. The team will combine this roll out with the implementation of central logging changes.

Central Logging

The team started working with a new central logging tool to replace the old one. The team is currently investigating the available options to capture Cardano application logs. With the Shelley code base, the team will move to having iohk-monitoring-framework log directly to the new logging tool.

Stakepool Tooling UI

The team has started designing the UI for stake pool operators to handle common tasks. The stake pool tooling will also support a fully command-line interface for scripting.

CARDANO DECENTRALIZATION

Research and Design

Work has been progressing on the formal specification for the transition from Byron to Shelley, and the team completed the first round of review this week. Some changes are required following the review, which the team will be working on in the coming week.

Development

Great development progress this week with the team completing the implementation of the header-body split, which they’ll be demoing internally soon. Previously, both header and blocks were downloaded together, whereas now headers are downloaded and validated separately before blocks are downloaded from only selected peers. The completion of this milestone marks the removal of the last prototyping simplification of the Ouroboros implementation, and thus the completion of the final production design.

Additionally, the completion of the header-body split also acts as an integration milestone for the consensus and network layer, with the consensus layer making more complete use of the network layer functionality, including the block fetch protocol and block fetch decision logic.

GOGUEN

This week the Plutus team started working on the specification for extended UTXO. They were also busy with enhancements and fixes for the Transaction tab in the Plutus Playgrounds, and a fix for the drag and drop functionality in Firefox.

This week the Marlowe team were working on sample tests and also general improvements in Meadow. Subscriptions in the online Udemy courses for both Marlowe and Plutus continue to grow, and the feedback continues to be positive. The Education team were working on translating the captions in these courses and on including subtitles with the help of the community. Work is continuing on the initial chapters of the Plutus ebook.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us. Please see the IOHK Careers page for more details.

Weekly Development Report

DAEDALUS

Wallet

This week the team finished the implementation of the 'Block consolidation status' and 'Network status' screens redesign.

The team has improved the user error message experience by implementing an automated scroll-to-error-message feature which is triggered upon form-submission if there are any errors thrown. By implementing this behavior, no error messages can be missed even on small screens.

A small issue with the Daedalus log file being saved with ".log" instead of the ".json" file extension has been fixed. The team has also improved the testing suite by adding unit tests for wallet recovery phrase generation and validation.

App Platform

This week the team improved container provisioning in the Docker-based Plutus execution engine and then moved attention to client-side concerns. A design for a JavaScript SDK to provide keypair and wallet submission was determined, to be used as a library for the smart-contract-js-client to depend on.

BACKEND OPTIMIZATION

Roughly a third of the team was engaged in training sessions in Miami last week, following the IOHK Summit 2019.

Meanwhile, the developers are nearing completion of a full transaction builder, signer, and submitter API. All of the core internal components have been finalized and tested, such as coin selection, fee calculation, key management, witness creation, and the transaction serializer. Some work still needs to be done on integration and comparison tests, which should be finalized next week. The API is getting very close to completion and now fully supports essential wallet management, transaction creation, and submission.

NETWORKING

The team has been working on new network functionality for Shelley, and on Friday a team member will be hosting an internal demo for smooth migration functionality from Byron to Shelly.

DEVOPS

Continuous Integration

The team continued work on integrating nix-tools with the new repositories, including cardano-shell and plutus.

Monitoring

The team continued with monitoring tooling migration and tested the changes in an isolated environment. The changes are being finalized for merging and will be rolled out to all environments by the end of the month.

CARDANO DECENTRALIZATION

There is no update this week.

GOGUEN

This week the Plutus team worked on integration tests for the backend, as well as updates to the specification and crowdfunding example. Updates were also made to the alignment of tabs in the Plutus Playgrounds, as well as an improved responsive display for parts of the user interface.

The Marlowe team were busy making syntax updates and general improvements in Meadow. Participation in the online Udemy courses for both Marlowe and Plutus continues to grow, and the feedback has been positive. The Education team are working on producing some supplementary information as part of this feedback. Content creation continued on the Plutus ebook.

ANNOUNCEMENTS

IOHK is currently looking for talented people to work with us. Please see the IOHK Careers page for more details.