Status Updates
October, 2019
October 25, 2019
WEEKLY DEVELOPMENT REPORT
DAEDALUS
Cardano Wallet
This week the team integrated the latest Cardano wallet backend and Rust node into Daedalus, which opened a chance to integrate network information with the V2 API endpoint. Since there were several changes to the API specs, the team had to update the implementation of already-implemented endpoints - mainly the ones for handling wallets and transactions.
In order to further improve performance, the team worked on reducing layout re-renderings by optimizing data handling.
In the scope of regular maintenance tasks, the team also worked on fixing the build mode of Webpack auto DLL plugin and speeding up Storybook builds.
App Platform
Full-scale performance testing of a mainnet Byron deployment of Cardano GraphQL was initiated using the four services of the stack: AWS RDS cluster, Haskell PostgreSQL write service, Hasura, and the GraphQL server. The initial goals were to identify performance bottlenecks, analyze SQL query performance, and determine areas that would benefit from optimizations. The results were used to make improvements to the underlying SQL schema in coordinating with the node client developer, and the interface improved to enforce more explicit query argument requirements.
Cardano Explorer
This week the team implemented pagination, which is used across the app, and worked on polishing the UI and optimizing the UI components structure.
A load test has been designed based on the application query profile and will be used next week to assist with planning the public deployment, in addition to establishing a benchmark to run as part of CI.
The API team also assisted with providing technical direction on how the full stack can naturally be applied within the Jörmungandr networks, adding value as a complementary application for the testnets.
WALLET BACKEND
Wallet
The team is closing the gap between the current Byron wallet implementation and the new Shelley implementation by providing a featureful API supporting both Byron and Shelley wallets at the same time. This will allow easy migration of Byron wallets to the new Shelley format (API documentation available).
The team is also adding extra features to make users' lives easier by pushing more information through the API. It now reports about things such as node's tip and height, network tip, and precise syncing progress of both wallets with the network and node with the network.
In parallel, efforts are ongoing on stake pool tracking and listing by pulling information from multiple sources (blocks themselves and the node) and combining them into a single digestible source for API clients like Daedalus.
NETWORKING
This week, the networking team has been working towards a production readiness of clients (explorer and node). For this, they extended the subscription worker to support local clients and updated the error policy patch. Both changes are in review.
The team has been working on asynchronous IO on Windows using the Windows API, and wrote a small demo application which shows that the multiplexer works on Windows using this approach. They also wrote channels that support delays based on ΔQ: a change that allows for improved tests, and enables the team to write more complex test scenarios of ouroboros-consensus.
The team also made small changes to the MonadFork, splitting MonadThread out of it, which gives access to two operations: retrieving thread ID and labeling a thread, which is useful for more debugging scenarios using GHC's event log. Lastly, the team is continuing the design work of the P2P selection mechanism.
DEVOPS
The DevOps team has been continuing their work on the Jörmungandr node.
CARDANO DECENTRALIZATION
The team continued to add generators and property tests. They now have the pool registration certificates and protocol parameters being generated and have new property tests using them. It took some work to make sure there were valid signals in the traces generated.
The team also gathered and organized examples to begin benchmarking the different shrinkers. In addition to property testing, the team now has proof sketches of why the properties are true. Currently, these are hand-written; they will be polished and put into a LaTeX document.
GOGUEN
This week, the Plutus team worked on ledger updates to pass PendingTx as a data type and make the specification simpler. The team also added an escrow to the contract. This works like the crowdfunding contract, in that contributions can be made in parallel. The Marlowe team changed the wording within Blockly to avoid misconceptions and to make the representation more compact. They also added symbolic execution for the Marlowe Playground and made some other minor updates.
October 18, 2019
WEEKLY DEVELOPMENT REPORT
DAEDALUS
Cardano Wallet
This week the team finished Daedalus newsfeed infrastructure setup. This was the last step in the preparation of the upcoming Cardano 1.7.0 release, which includes Daedalus 0.15.0 and Cardano SL 3.1.0.
Work continued on the desktop installers for the Jörmungandr testnet, with further progress now dependent on the cross-compilation of the wallet service. The team created a nix-free way to launch Daedalus in a frontend-only mode.
A note has been added to the 'Wallet UTxO Distribution' screen. This informs the user that pending transactions may affect the accuracy of the data presented on the UTxO distribution chart. The note also displays the number of pending transactions a user has in his wallet.
The 'Delete Wallet' option shown on the 'Wallet Settings' screen has been redesigned to fit better with the newly added 'Wallet Recovery Phrase' section. The 'Delete Wallet' link has been replaced by a 'Delete Wallet' section, which provides more information on this option and highlights the importance of knowing a wallet's 12-word recovery phrase before it is deleted.
In the scope of regular maintenance tasks, the team worked on fixing minor styling issues on the Cardano theme and optimization of the flat button colors. The react-polymorph 'Dropdown' component has been optimized to better fit the requirements and usage-patterns of the Daedalus UI.
App Platform
No update this week as the team has been focused on the Cardano Wallet and Cardano Explorer development.
Cardano Explorer
This week the team finished the implementation of the design for the 'Transaction Details' and 'Stake Pools List' pages. This is an important milestone as now all the key UI elements have been completed for implementation within the Jörmungandr Explorer App, with the initial version to be derived from the example application contained within the js-chain-libs repository. This makes full use of the GraphQL API exposed by Jörmungandr and will serve as a baseline to continue development in parallel with the Byron Mainnet explorer.
WALLET BACKEND
It has been an intensive week for the wallet backend team. The team's efforts in testing rollbacks and chain-following have finally paid off. While part of the team was busy with making sure the wallet worked in a decentralized setup - where nodes may switch to different chains arbitrarily - another part has focused on getting data about stake pools stored in a database and served through an HTTP interface. This will enable frontend clients like Daedalus to give insights about available stake pools and how they perform.
We are also continuing our efforts to introduce a backward-compatible API to effectively allow current users of Daedalus to migrate their wallet into the new format now supported by the backend (the same one used by Yoroi).
NETWORKING
Last week, the network team started exploring the Windows overlapped IO API to get reliable support of IO on Windows. The team also integrated the latest space-leak fixes and performance tweaks in the block-fetch protocol into the cardano-node repository, and provided an ARM how-to: useful for people who would like to build and run a cardano-node on RockPi. Lastly, the team started polishing error policies, which are important for making the node - and clients - production-ready.
On the P2P front, the team worked on network topography generators and are looking into interactive plotting solutions for Haskell. Part of the team also met at a major Haskell conference. IOHK had a solid presence, IOHK colleagues presented five (four by the networking team, the other related to IOHK's work on P2P) highly valued talks, which can be found online:
- IOHK's session-type framework
- State machines and categories
DEVOPS
DevOps has been working hard on getting installers working for Daedalus that integrate with the Cardano wallet and Jörmungandr. The DevOps team has also been refactoring deployments for Jörmungandr and Cardano, to make it easier to deploy production clusters.
CARDANO DECENTRALIZATION
The team updated the spec with a new mechanism for bringing the rewards from the incentivized testnet onto mainnet. The team have also completed writing more generators, and are close to having the ones needed for the ledger/transaction level. However, there's still work to be done to get to the top chain level, where the team is generating block headers, etc. The team also added more high-level properties to the plan and made progress on the Byron-annotated serialization. They are now down to fixing a few test failures.
GOGUEN
Last week, the Plutus team made improvements to logging and error handling in contracts, where they added an error constructor to deal with errors that happen during contract execution.
The team also updated the currency contract and improved the pubkey contract, and made changes that enforce the correct type for the ValidatorScript.
The Marlowe team made further updates to the Marlowe tutorial, to highlight the differences with previous versions. They are also preparing for the new version of the Udemy course for publication very soon.
October 11, 2019
WEEKLY DEVELOPMENT REPORT
Daedalus
Cardano wallet
This week the team finished the integration of the available V2 API endpoints, which will enable the support of Shelley features. Work is continuing on modifications to the automated end-to-end tests so that they can be used to check that the API integration is working as intended. Work also started on new desktop installers for the upcoming incentivized testnet version of Daedalus.
In the scope of regular maintenance tasks, the team finished working on reorganizing Storybook stories and improving the automated testing suite to better match the Daedalus UI structure.
App Platform
The PostgreSQL views were further optimized this week based on some performance issues which were discovered during testing. The improved views have been submitted to the upstream repository Cardano Explorer, making the interfaces available for all database users. Hasura configuration is also incorporated into the code repository now, completing the hands-off setup of the stack.
The first pre-production version of Cardano GraphQL was also released this week for the Explorer team to use as the target for the Byron mainnet.
Work on Jörmungandr integration for the incentivized testnet version continued, establishing a Docker image and inclusion in the project’s stack.
Finally, testing infrastructure has been set up to stress-test Cardano GraphQL against a fully-synced Byron mainnet data source, where the scaling strategies for hosted deployments will be determined. A scope was also determined with the Plutus team for the chain publishing node client.
Cardano Explorer
This week the team finished the implementation of the address details and block details pages and started working on the transaction details and stake pools list pages. The team has also implemented theming support, which is an important step as there will be multiple Cardano Explorer instances (one for each network), and each will have a different theme to distinguish it from the others.
Wallet backend
The team is now working on three fronts simultaneously: Byron wallet support, stake pool tracking, and chain following in a decentralized network where forks may happen arbitrarily. The latter two are finishing up and are getting most of the team’s attention in the form of testing.
The team has also reworked the networking layer to reflect that nodes may switch to different chains as they see fit. The change was handled in such a way that the logic can be implemented for Jörmungandr while leaving the overall interface sufficiently flexible to allow implementation using a different networking interface, such as the Haskell cardano-node.
Finally, the team is now also collecting data about stake pools and returning a list of existing stake pools with the number of blocks they are producing. In the next iteration, a performance metric will be introduced to allow clients to list and order stake pools according to how well they perform.
Networking
This week the team spent time planning work for the upcoming peer-to-peer component and has already started implementing the first aspect, which is graph simulation. Soon the team will be able to develop and asses local policies for nodes that will govern which nodes should and shouldn’t be connected to. This is the first time such policies have been created for a globally-operating proof-of-stake system, and the team is being especially careful in the evaluation process of the policies.
In other news, after the pipelining of the chain-sync protocol was included, memory consumption of the node was reduced, and block-fetch protocol constants updated, the team achieved a 6.97 Mb/s download synchronization speed for the Shelley node. As a result, the team was able to synchronize the current blockchain in just fifty minutes.
Finally, the team is finishing off work on an implementation of a pure STM monad (software transaction memory), which includes the last missing orElse operator, together with a test that translates the formal specification of STM. The team was able to validate their implementation with GHC's implementation and haven't found a single bug across millions of iterations of QuickCheck property tests (that is, each property test was run over millions of randomly generated STM transactions).
DevOps
This week the DevOps team updated the Snappy and Nix scripts for the 0.5.6 networked testnet release, as well as creating a new Chocolatey package for 0.5.6. A new node cluster has been created so that the team can do some performance testing of the testnet network, and a new tool has been added to the jormungandr-nix repository to analyze block production over both individual epochs and the entire chain.
Cardano Decentralization
Using the infrastructure developed to detect space leaks, the team was able to fix all space leaks in the node this week, resulting in lower and more predictable memory usage.
Goguen
This week the Plutus team added a new UtxoAt query to allow contracts to query the UTXO set through the node client. This changes the way the WalletAPI is implemented in the emulator as wallets now keep track of the entire UTXO set, not just the subset of unspent outputs at the addresses in which they are interested.
The team also removed the Sealed type and added a generic Data type for use in contracts. In addition, they simplified how script outputs are represented to make them more consistent with how they are handled in PlutusTx. Changes were also made to make the writing of parsers easier.
The Marlowe team updated the Marlowe tutorial to include a new section about static analysis. They are also preparing for the new version of the Udemy course for publication soon.
October 4, 2019
WEEKLY DEVELOPMENT REPORT
DAEDALUS
Cardano Wallet
This week the team finished the 'Newsfeed' feature which was the final addition for the upcoming Cardano 1.7.0 release.
The team has continued working on the V2 API integration which, will enable Shelley features. Both the transaction fee estimation and the create transaction endpoints are in the code review and testing phase now, while the team is continuing to work on the implementation of the transaction history API endpoint.
App Platform
This week the team has been working on a remodel of parts of the Cardano JavaScript SDK so that they will be compatible with the Shelley era, ensuring that both Jörmungandr and Haskell nodes will be compatible too. The js-chain-libs library will be used to provide the client-side behavior for the smart contract backend, namely the wallet for coin selection, and then the interface for submitting to the network.
In other work, a simplification of the extended UTXO-compatible node client was determined this week and will be specified next week in coordination with the Plutus team. This refinement aims to make the node client more powerful and more generic, keeping the state and logic relating to triggering Plutus contracts contained within a single service. Its primary behavior will be a publishing service to enable WebSocket clients to follow the chain.
Finally, the git-based stake pool metadata service was updated to handle remote synchronization this week, and now awaits implementation within the API layer.
Cardano Explorer
This week the team finished the implementation of the design for the epoch details page and continued working on the address details page, as well as the block info component which will be reused across many pages.
The Cardano GraphQL Hasura implementation was configured to work with the Postgres views developed last week, and performance tuning was done. This enables query delegation from the public-facing API after a query is deemed acceptable and passes the validation rules. The views will be transferred to the Cardano Explorer database definition after the work is finalized.
NETWORKING
This week the networking team implemented interruptible foreign function interface (FFI) calls for the Windows named pipes API. Early tests are promising. The team also added support for mixed IPv4/6 addresses and tested it on a small network, in which some of the nodes were running on RockPi machines. There was also a bug fix in cardano-sl subscription management as well as improved performance of the cardano-byron-proxy.
The team was also working on propagating chain sync changes this week, which provide a significant performance boost, especially when syncing a new node and downloading the whole blockchain. Work is continuing on cardano-node benchmarks, and the team has started working on one of the final peer-to-peer components required for decentralization.
DEVOPS
This week the team was busy updating the Snappy and Nix scripts for the Jörmungandr testnet. A Prometheus exporter for Jörmungandr has also been created to get metrics for a stake pool. Finally, the team has worked on the Chocolatey package for Jörmungandr 0.5.5, which is now available.
CARDANO DECENTRALIZATION
This week the team worked on restructuring the codebase, and most backends are now implemented in their own libraries, which will help projects that integrate this framework to manage dependencies. The project's dependency list will only grow along with active backends included. The team has also started to implement a benchmark for chain syncing that could be turned into a nightly continuous integration job.
The transaction generator is now complete, and the team has started to obtain results from benchmarks that now need to be extended to a distributed system benchmark.
GOGUEN
This week the Plutus team made improvements to the API for payment selection. This improvement gives users an easy way to write contract code where the value spent is specified in several iterations, giving more control over the inputs that are used by the wallet (for example, to cover fees). The team also updated the documentation for the ledger interval module.
The team also worked on blockchain visualization changes in the Plutus Playground, including the display of brought forward balances in a table so that users can click on inputs and outputs to jump to their related transactions. Other improvements included some general tidying up of on-screen elements as well as the removal of redundant code.
The Marlowe team presented the new version of Marlowe (version 3.0), at the Cardano anniversary event in Bulgaria last weekend. They also demonstrated the Marlowe Playground and its associated tutorial. They are also preparing the new version of the Udemy course for publication soon.