Download the PHP package getinstance/listingtools without Composer

ListingTools

CLI tools for indexing, extracting, and embedding code listings into markdown article or chapter documents. Originally dogfood code written for the book PHP Objects Patterns and Practice (Apress).

ListingTools helps you to

• keep your sample code in a runnable and testable state
• find all listings easily in your code archive
• insert code examples into your article, blog post, or chapter
• reimport updated/improved listings easily
• renumber and reflow code examples to accommodate additions and deletions

Installation

There are three easy options

Install as a project with composer

If your current project does not have a composer file (and you don't want to create one), run composer create-project to get the tools in a subdirectory.

This will generate a listingtools directory. You can invoke the tools from anywhere.

Download and unzip

You can find the latest release at https://github.com/getinstancemz/listingtools/releases

Download and uncompress the source code archive and run as above.

NOTE Since the dependency won't be enforced for installation using this method, it is important to be aware that the project requires PHP 8.0.

Add to a composer-based project

The tools will be added to your vendor directory. Scripts can be accessed via vendor/bin

NOTE Because the scripts are installed as composer binaries you do not need to explicitly invoke PHP.

Quick start

Add comments to your source repository defining the code blocks you wish to extract. You can use slash-star comments although hash comments, HTML comments and a custom json element are also supported

Create content slots in your chapter/article

Run the gencode.php command to generate the code blocks and insert them into your manuscript (always do this after committing your work so that you can roll back if necessary).

Those arguments are: an arbitrary namespace for your project (this is required but only used with the GitHub gist feature), the directory of your source repositiory, your article or chapter, your output file. If you do not specify an output file then the command will write to STDOUT.

Your code slot will then be filled with the corresponding code as marked in your code comments:

As you can see here `getMatches()` will give you access to found listings

<!-- insert 001.01  -->

<!-- endinsert -->

When client code calls this...

If you need to improve your source code, fix it in the repository and not the manuscript, then run gencode.php again -- your listings will be updated.

The tools

The quick start section demonstrates some useful functionality, but it also begs some questions. How do you insert a new listing, for example, without having to hand-renumber all the listings in your chapter? For my book, some chapters contain approximately a hundred listings -- inserting three or four new listings early in the chapter would have meant... well it's the kind of work that any programmer would rather automate than do manually -- which is why I'm writing this document now. Anyway, here are the details.

doindex.php

Generate an index of all listings marked in the referenced repo

Arguments

Side effects

None. Entirely read only. Does not write a cache.

Notes

This command provides a useful overview of listings during development -- ordered by article.listing number. Can also be usedd to generate an index for readers where a code archive is to be offered alongside a publication.

Example

gencode.php

Read the source listings and the manuscript. Match the source listings with the corresponding manuscript slots. Output as directed.

CAUTION This command can change your files quite extensively depending upon how it is run. Always commit to a version control system before running.

Arguments

Flags

Side effects

Where an output argument is given, may write extensively to the specified file (unless the -d flag is used). If the experimental -g flag is used, then the listing code is created or updated as a gist, and the output slot will be given the corresponding gist embed.

Notes

This is the business end of ListingTools. It is how the code gets copied from your source repo (which should be the source of authority for code) and into your manuscript. When using it to write output to a manuscript file exercise extreme caution. Version control, and not this tool, is your reset button.

Example

In dry run mode, the command ouputs a list of matched listings <-> slots but takes no further action

nextlist.php

Given a chapter or article number, work out what the next listing tag should be

Arguments

Side effects

None. Entirely read only. Does not write a cache.

Example

To find the next listing in the current directory:

output.php

Given the source directory and a listing number, collate the listing and write to standard output

Arguments

Side effects

None. Entirely read only. Does not write a cache.

Notes

A useful way to check what a listing will look like when broken up within a file or even spread across several files without having to first generate and embed in the manuscript file

Example

renum.php

Renumber all listings in the given source directory following sort order so that they are contiguous. This is a good way to insert new listings.

CAUTION When run in anger (ie without flags to suppress or redirect output) this command acts recursively on the given directory, potentially altering many files. Never run this on a repository that cannot easily be rolled back to a previous state.

Arguments

Flags

Side effects

Potentially very large. Will recurse through files in the source directory and renumber listings. Always back up before running.

Notes

Typically you would use this to handle deletions or additions. Imagine, for example, your listing index looks like this:

During development we might remove 001.01. Then we might decide we want to add a new listing before the first. To do that we might create a listing tagged 001.00.01. Now the index looks like this:

In the manuscript file we should keep slots up to date with listings in source. Finally, though, when we want to clean up our numbering to close gaps and remove additional listing tag clauses, we can run renum.php:

At this point you should run git diff or equivalent to confirm the sanity of the process. Then you can run a gencode.php reflow (which inserts listings in sort order ignoring and then updating the stipulated slot tags.

Assuming that your slots and listing count match this command should reimport and retag your newly renumbered listings.

Tagging the source code

The listings within a source code repository are defined by start and end comments. Typically these use hash-star notation:

ListingTools also supports hash comments:

And, in order to allow snippets of JSON:

JSON listings are relative new at the time of this writing, so should be treated with some caution.

One listing, many tags

Real testable code often needs to contain much boilerplate. In order to be runnable, a class may need multiple getters and setters, error checking, persistence logic that is not essential to the demonstration at hand. You can cherrypick the code you want to include in your listing by opening and closing your listing tag multiple times within a source file:

When compiled listing 001.02 looks like this:

Interleaving listings

Code often appears more than once in an article or chapter. You might, for example present a rolling example in which you demonstrate the parts of an althorithm, and then wrap up with a final 'putting it all togeter' example. It is much cleaner to use a single soure file for this than it is to have one file for the parts and another for the combined listing. ListingTools allows you to embed listings within listings. It will ignore and suppress listing tags that are not relevant to the listing tag at hand:

So there are three listing tags at work here. Listing 001.05 gives

Listing 001.06 gives

Listing 001.07 provides the 'let's bring it all together' listing:

NOTE I did not close the tag for listing 001.07. ListingTools will happily include the rest of a source file if the opening listing tag does not have a corresonding close tag.

Listing tag flags

This (new and experimental at the time of writing) feature allows for some presentation hints to be added to listing tags. Currently supported flags are:

So for this code block:

The output will omit the trailing newlines.

You can combine the flags. Like this:

This will produced a listing which reconstructs well-formed JSON:

The manuscript listing slots

Obviously, defining listings is only one side of the equation. The next stage is to define the slots into which the listings will be imported by gencode.php. These are simply HTML comments that look like this:

Unless listing 001.01 is found, gencode.php will throw an error (see above for reflowing -- an exception to this rule). Otherwise it will insert the listing where specified:

<!-- insert 001.06  -->

<!-- endinsert -->

Finally

This documentation v1 brought to you with the support of having-covid-on-vacation. It's great. Don't try it.