`dds` Alpha Release 5 is Available!

The fifth alpha release of dds is ready for public consumption!

Changes in Alpha 5

This release is a small, incremental improvement over past versions. A lot of work is still going on behind the scenes to support dds, even if the main repository isn’t receiving a constant stream of commits and issue updates.

Features and Improvements

Packaging: `sdist` and Gzip+Tar

The vast majority of work in this release was in adding Gzip and Tar support. dds now knows how to import/export packages as Gzip’d Tar archives. While this isn’t immediately beneficial to end-users, this is in preparation for network-based distribution of packages and catalog data.

Toolchain Option: Runtime Library Mode

dds toolchain files can now declare their desired runtime and standard library settings. For now, this is just selecting between being static or dynamic, and choosing to use the debug or optimized version.

With MSVC, the static runtime is chosen by default, and if the /runtime/debug setting is unspecified, it will be inherited from the top-level debug option. This improves over dds previously specifying the /M flag at a lower level, which made it difficult to modify.

On GNU-like compilers, the /runtime/debug setting will never be enabled automatically, since most people aren’t used to working with it on those platforms. This may change in the future. Enabling /runtime/debug on Clang or GCC will set preprocessor definitions that enable the debug-mode for the standard library. It should be emphasized that translation units compiled with debug-mode standard libraries are not compatible with translation units compiled against a non-debug version of the standard library. While MSVC has safeguards and features to prevent this combination, GCC and Clang lack such features.

Toolchain Options: Split Debug Info

dds toolchain files can now request split debug information. That is, a mode in which debug information is stored separately from the main compiler outputs. This can greatly shrink link times and reduce the size of generated binaries.

Note: While MSVC has supported this for a long time, other compilers are expected to support the -gsplit-dwarf compiler option.

Note: Split debug info with MSVC does not yet specify an output file for the split debug information, which means that all debug information is collected into a single file adjacent to the object files. This will be changed in a future release to emit a single debug-info file next to each corresponding object file.

Subcommand: `dds compile-file`

One can now request that dds compile a single file in a project rather than the entire project at once. This is meant to help diagnose individual compile errors and in the future assist external build drivers (such as IDEs or more complex build tools).

Miscellaneous:

dds now has a --log-level option to enable more verbose logging.
The published Linux binary is a static executable, removing need for any runtime libraries (including glibc). This may be changed in the future.
On Windows, dds now sets the codepage of the console to UTF-8, fixing rendering of UTF-8 encoded text that it writes to the console.

What’s Next?

Currently, the main goal is to get dds able to download packages and receive catalog updates over the network. So… What’s taking so long?

I’ve been bootstrapping and “dogfooding” on dds from the very beginning, and I’ve been writing a lot of libraries to support my efforts in the interim. This last phase of development was largely focused on Gzip and Tar archive support, which were written all by hand, with the aide of Zlib for DEFLATE compression.

What’s Been Done?

To give some perspective on the scale of the work, here’s the libraries that have been written over the course of building dds to where it currently stands:

pubgrub: A new C++ implementation of the Pubgrub dependency solving algorithm
neo-buffer: A data buffers library
neo-compress: A data compression library, including an implementation of Gzip and ustar
neo-sqlite3: A SQLite 3 API wrapper
vob-json5: A JSON5 parser
vob-semester: A generic semistructured data processing library
semver: A library for handling Semantic Versions
neo-fun: A set of library fundamentals and utilities
neo-concepts: A polyfill implementation of most of the <concepts> header for GCC 9

(One might refer to the above as “yak shaving.”)

None of the above libraries are dds-specific, and all of them are liberally licensed, and I hope that others will be able to find them useful. Some wrap other libraries, and I’ve used other libraries as part of development that aren’t listed above, and I am incredibly grateful to those and their developers.

What’s Left?

To meet the next milestone of network package and catalog distribution, I have a few things I must have before I will be satisfied with the feature:

neo-http (no GitHub yet) will implement various HTTP functionality in terms of neo-buffer without any I/O involved. (This will not include TLS.)
A method of distributing incremental catalog updates and package data over HTTP.
A yet-unnamed cryptography library, to provide file signing. (Wrapping libsodium.)

With these changes, we’ll begin to see dds capable of downloading packages and catalogs from network locations.

dds Alpha Release 5 is Available!