Forgotten Engines

 

The warrior wields a raygun. The conjurer casts Knock to let the sapient trilobites out of the cryo-chamber. The thief has a plan to trick a dragon into fighting the crazed robotic lizard that's currently trashing the facility.

 

Forgotten Engines adds confused Neanderthals, ancient fungoids, and shiny hoverpacks to your swords-and-sorcery roleplaying game.

This 65-page supplement is crammed with ancient advanced technology and time-displaced beasts. It's designed for use with The Black Hack – or you can adapt it for any old-school tabletop RPG.

As if you needed a justification for putting robots and stasis fields in your fantasy setting, Forgotten Engines provides a history of those prehistoric super-intelligences, the Desperate Fungoids, and the flight of their civilisation ever further forwards in time in an attempt to escape their doom.

 

Thirty techno artifacts, from robotic exoskeletons to deployable containment chambers, railguns and rayguns for all occasions, and the Magnificent Autodibble!

Prehistoric beasts: mammoths, fungoids, proto-troglodytes, nautiloid hiveminds, uplifted terror lizards, cyborg apes, and psychic tree ferns!

Random tables for robots of all shapes and sizes!

A surprising number of ways to be hit by mega-laser blasts fired from the moon!

 

To help create adventures, there's:

Setpiece ideas like "out-of-control robot tank guarding a mothballed munitions factory" and "crashed geodesic biodome sinking into swamp, systems running down in odd ways"!

Random site modifiers: Escaped aliens! Mutagenic gas! Gravity free zones! Caveman uprisings!

New spells and odd NPCs to be found in Desperate Fungoid city ruins!

 

Includes a Creative Commons license for hacking, and an alternative PDF for small screens and home printing.

Available now:

https://periapt-games.itch.io/forgotten-engines

https://rpg-trader.com/products/8864/forgotten-engines

https://www.drivethrurpg.com/en/product/581529/forgotten-engines

One more generative AI rant for the pile

(this one's about summarising text)

LLM chatbots – that is AI, in the same sense that we could just start saying "doctors" to refer specifically to orthopaedic wrist surgeons if we collectively decided to – 

LLM chatbots continue to slosh about the world. I used to try them out intermittently to see if they were any good.

My contact with the technology is only incidental these days. To wit:

• If you google old phrases and terminology in English, a LLM chatbot will still confidently weigh in with completely spurious "definitions" because they're not well-represented in the training data.
• If you google modern bits of even slightly less-discussed technical knowledge like "does a Kickstarter project video appear on the prelaunch page", a LLM will still confidently tell you the opposite of the truth.
• If you need customer support or anything that even looks like customer support, there is an extra quarter-hour minimum of wasted bot effort before you can get it.

Nothing I've seen has suggested the technology has fundamentally changed.

 

He can't be wrong, he writes so confidently.

 

In the previous edition of discussing the emperor having no clothes, I mentioned  

[Wikipedia] editors pointed out that the LLM summaries generally ranged from 'bad' to 'worthless' by Wiki standards: they didn't meet the tone requirements, left out key details or included incidental ones, injected "information" that wasn't in the article, and so on

and 

bureaucratic wonks note that genAI can't summarise text. It shortens it and fills in the gaps with median seems-plausible-to-me pablum. The kind you get when you average out everything anyone has ever written on the internet.

I recently saw an AI booster shuffle their position back to "at least it's good for summarising, it's going to completely replace human effort there". With that motivation, let's drill down a bit.

In (a) summary

Let's not bury the lede. LLM chatbots can't produce good summaries. Sometimes by chance yes, but not reliably. Summarising, like everything, is a skill-based task, and of the various capabilities required to do it well, LLMs lack four of the most important.

1. LLMs won't reliably retain important structure or order in which information is presented. They will just haphazardly obliterate implicit linkages. They will even occasionally discard explicit structures, as when the text itself points out that C follows from A and B, and therefore D.

2. LLMs can't identify the most important information in a text (a necessary first step to preserving it in the summary). In a good summary, certain content "should" be retained, certain content compressed, and the remaining content discarded. Vital information generally isn't identified within the text in a way that's detectable without broader context, language skills, and understanding of the world. Even when it is, e.g., in texts where repetition of a word corresponds directly to importance, or phrases like "this is vital information" are always appended, LLMs still aren't guaranteed to retain important details! And the same applies to cutting out unimportant information.

3. LLMs can't stick to the source text, that is, the content they're meant to be summarising. Because they just generate text (by predicting which bits of text should come next, based on an enormous model of which bits tend to come after which bits, hence 'language model'), there's no internal representation of Things 'In' The Language Model versus Things 'In' The Text To Be Summarised, and no impetus to perform computational operations that keep them separate where appropriate. All of which is to say that as well as not including things that should be in a summary, an LLM will readily include things that shouldn't be. Oops

3(corollary). That includes things that aren't true. Oops(corollary)

4. LLMs will sometimes just negate statements for no clear reason. When processing text, e.g. when directed to "summarise", they'll turn a claim into the opposite claim. I think what's going on here is that a statement and its negation are syntactically and semantically similar, even though their meanings are devastatingly dissimilar. Too bad LLM technology doesn't get meanings involved, instead just taking a probabilistic walk through a model of language features like, oh I don't know, syntax and semantics!

Note what these four crucial capabilities have in common. It's the reason why LLMs can't do them. That's right, they require understanding to do properly.

Or if not understanding, then at least computational models of understanding, like formal reasoning over symbolically-encoded domain knowledge including useful axioms. I mention this because classic AI systems (planners, searchers, problem solvers, etc) can do just that, in their various limited ways. They symbolically represent domain information and then perform operations on those symbols which can then give something potentially useful back once related back to domain information.

And those systems are limited, yes, but LLMs don't do 'understanding' at all. As far as I can tell, on the back of a postgrad compsci degree and a few days spent reading and partly understanding the computational basis, this is a fundamental limitation of the technology. One which can't just be fixed, but which would need a whole new (at most LLM-inspired) technology to overcome. For exactly the same reason why AI "hallucinations" can't be fixed.

 

Presummary (a digression)

This technical basis of how LLMs work also explains something else. These chatbots are particularly bad at "summarising" documents which contain surprising content.

By surprising content, I mean...

➡️ Statements seeming to defy common wisdom. Things that are the opposite of statements well-represented in the training data. When X is generally true of a field, but your text describes how ¬X is true of some narrow subfield or specific context, you'll see an LLM "summarise" X into ¬X more frequently.

➡️ Deliberate omissions of things that are usually in correlating training data documents. If your text looks like a text of type blarg, and blarg texts in the training data typically report on X, but you have not reported on X for your own reasons, an LLM is likely to just make something up about X while "summarising".

➡️ Unusual pairings of form and content. Performance degrades the more you ask an LLM to do something novel.

➡️ Context-sensitive language like metonyms and homographs. When X is a big important noun well-represented in the training data and X refers to something else in the text, you'll see an LLM (appear to) get confused by the statements about X its produces for the "summary".

➡️ Nontextual information content. The LM stands for language model. If you have a report that includes and discusses images and diagrams, a chatbot might be able to stop and parse those, and then incorporate its own description of the image as part of the text to be summarised, and maybe even put images back in the summary. But you'll nonetheless end up with a worse output.

 

In summary (but for real)

So LLM chatbots can't be (consistently, reliably, etc) good at summarising.

Of course people who don't know what a good summary is might not notice this; likewise people who possess the skill but don't carefully check the job they told it to do.

(I would argue that in either case, if the task was worth doing to begin with, you should prefer the task not getting done to having no idea whether your document is a good, adequate, or terrible summary)

Anyway this is why you may have seen people who do know what a good summary is point out that LLMs actually "shorten" text rather than "summarise" it. I'm not certain but I think the first time I saw this was in one of Bjarnason's essays.

The sentiment "this technology sure can't do [thing I am skilled at] for shit, but I guess it might be good at [thing I don't know about]" will continue to carry the day as long as people let it

I'll self-indulgently close by quoting myself again:

A lot of people with a lot of money would like you to think that genAI chatbots are going to fundamentally change the world by being brilliant at everything. From the sidelines, it doesn't feel like that's going to work out.

Trying not to be a Gell-Mann Amnesiac

I sometimes wonder how much Gell-Mann Amnesia people experience. Paraphrasing Crichton, when you're a domain expert, you'll sometimes read an article that gets every aspect of your field completely and absurdly wrong, have a little laugh about it... then keep on reading and trusting articles that are about other fields, even from the same publication or writer.

As if they're some pure spring of wisdom which only coughed out a lump of mud when it came to the thing you happen to know about.

It's just an idea from a novelist, not the kind of cognitive bias that's supported by real-world studies that I know of, but you have to admit that it has a kind of... truthiness to it.

Stack this up with Dunning-Kruger and it's easy to become cynical. You might decide that actually, all the loudest voices are talking complete nonsense, all of the time. That might be too far. But I do think it pays to put deliberate hard effort into distinguishing domain experts from overconfident bullshitting pundits.

Now, anyone with their ear to the ground and a weather eye out for Gell-Mann Amnesia should have arrived at the obvious conclusion about generative AI. To wit, that the current state of the technology is that it is an overconfident bullshitter.

On being a piece of software and being confidently wrong

The case studies are easy to find, and the ones from domain experts sound pretty different from the ones from the tech industry and the reporters too busy and/or demoralised to do more than repackage their press releases as articles.

➡️ I am not a historian. The historians I've read say genAI gets softball history questions mostly right and deep ones mostly wrong. Sometimes subtly, sometimes dramatically. It just makes things up when the evidence is scarce. It makes errors of commission and omission as well as having misplaced focus and drawing weird conclusions from premises.

➡️ I am not an artist. The artists I listen to say genAI art looks bland and awful and organic because it doesn't understand composition or anatomy or separate objects (because it doesn't 'understand' anything). It can't make an image that isn't well-represented in the training data, like a camel and a steampunk automaton jousting from the backs of sumo wrestlers. Same in other kinds of media: filmmakers say genAI can't do film because it can't take direction or keep track of characters or have a consistent shot.

➡️ I am not a Wikipedia editor (except incidentally). Earlier this year there was a wretched moment when the Wikipedia editors were going to have genAI article summaries foisted on them, although I think that's turned around now. The skilled editors pointed out that the LLM summaries generally ranged from 'bad' to 'worthless' by Wiki standards: they didn't meet the tone requirements, left out key details or included incidental ones, injected "information" that wasn't in the article, and so on.

➡️ I am not a manager. The managers say genAI can't even collate timesheets reliably.

➡️ I am not a novelist. The novelists say a genAI book reads like a statistical summary of all creative writing anyone has ever done, including all the embarrassing teenage fanfiction. It sucks at originality. And because it doesn't have an internal model or understanding of its outputs, it can't keep track of things and make a coherent satisfying story. Things are vague, tropey, or contradictory.

➡️ I am not a lawyer. The lawyers are, um, well, by the sound of it a lot of them are being sanctioned for using generative AI to cite completely nonexistent caselaw. (☉__☉”)

➡️ I am not a public policy wonk. The bureaucratic wonks note that genAI can't summarise text. It shortens it and fills in the gaps with median seems-plausible-to-me pablum. The kind you get when you average out everything anyone has ever written on the internet. If you try to have an LLM summarise or draw conclusions from a study, it will usually do a bad job, fabricating statements more along the lines of what an average person would guess if they'd only read the study's title.

➡️ I am not a software engineer. The software engineers seem to have mixed opinions. They say that genAI works as code autocomplete (something that has existed for fifty years, but this new kind has pretty sophisticated lookahead, neat). At least some are saying it can't do principled software engineering, it introduces security flaws, its performance drops off for obscure languages, it overconfidently generates bad code, it plagiarises from code repositories that it doesn't have the rights to...

I could go on.

I'm no longer a domain expert in anything, this many years after my stint in academia. I think I'm halfway to being an expert in a few different areas, though. I deliberately concocted some thoughtful questions at the intersection of those areas, just to see.

For example, I asked about the (obvious) mapping of choose-your-path text adventure books onto mathematical graph structures, which the LLM chatbot identified. I followed up with technical questions about the features of those graphs in context: what would the game be like if they weren't digraphs, would you expect cyclic vs acyclic, would a finite state machine be more appropriate and if so why, etc.

And lo, the generative AI output was absurdly, hopelessly, and confidently wrong when given questions that needed expertise.

A lot of people with a lot of money would like you to think that genAI chatbots are going to fundamentally change the world by being brilliant at everything. From the sidelines, it doesn't feel like that's going to work out.

Sometimes I read posts from experts along the lines of

"I've noticed it's almost worthless at [my field], but it sounds like it's pretty useful for [other thing]."

But less so lately, maybe?

So I'm left wondering: are people experiencing massive Gell-Mann Amnesia about these chatbots? Or does everybody know that the emperor has no clothes?

(But oh no, we've invested so, so, so very much money into the emperor's finery, and all the wealthiest people at the imperial court agree: pleeeease could you keep squinting to see this amazing new clothing?)

 

On becoming technologically competent

I was working out a computer problem and got to thinking about what I was doing.

Pretty much anybody who uses a computer on the regular would benefit from becoming more of a power user. Most people would like to work more efficiently to free up time for the things they want to do. Luckily, it's never too late to learn.

How, though?

You can just take it one step at a time, applying the miraculous human reasoning power of 'breaking a complex problem down into components which can be tackled'.

1. Identify a laborious part of your workflow.
2. Find a tool that will help.
3. Learn to use the new tool.
4. Add it to your workflow.

If a component task seems too onerous, you can break it down further.

For example, if 'learn to use the new tool' seems imposing, it is possible to:

• plan how to approach it
• learn to read documentation
• find a tutorial which works for you
• set time aside to read documentation and follow tutorials
• search for specific information in today's search-hostile web
• establish what the best forum is to ask questions in

Finding tools

Being a power user is not a binary thing. It's a gradient. When I first used a computer back in the dusty old yesteryears, I was not competent with it. Now I'm competent in many technological domains, and an expert in one or two. Some of that was acquired naturally and some through deliberate effort.

Right now, if I ever need to write a batch file or mess with the registry or write my own css, I have to re-familiarise myself, because I've let pieces of competence degrade. For some other tech questions, I still have no competence at all. Fortunately, it doesn't really come up, because...

Having a genuinely unique problem is vanishingly rare. 

You can be pretty sure that lots of clever technical people have already come up with solutions for whatever it is that you're doing. It's just a matter of finding their solution, learning to use it, and then actually using it.

There's tons of (free!) software which gives you batch processing or shortcuts or streamlining or other efficiency gains for all manner of otherwise-laborious tasks. It's a matter of identifying the need, discovering the tool, and learning its use.

Need to impose a complicated naming schema on a bunch of files? There's Bulk Rename Utility.

Need to strip audio tracks from videos? VLC does it.

Running out of disk space due to a regrettable history of erratic, poorly-labelled, manual backups made in arbitrary locations? SpaceSniffer + Duplicate Files Finder.

I could go on. I have bucketloads of discrete tools that I use either regularly, as part of my workflow, or intermittently, as part of problem-solving or error remediation or unusual tasks.

(We unfortunately live in the 'app era', the 'cloud era', the 'SAAS era'. People seem not to think of software programs as tools that you download and use to improve the ease, efficacy, or efficiency of specific things they do. I think they're missing out.)

Living means learning

If you put your mind to sliding yourself up the 'power user' scale, you'll almost certainly succeed, and you'll find that you can improve your workflow incrementally. As you learn and develop expertise with your tools, you will learn their best use cases, and find yourself getting judicious, and seeking more methods, more approaches, more tools.

A long time ago I set down the largely-worthless Windows file management tools, and picked up Everything and SyncBackFree. I learned mass image editing back in university using some tool I don't even remember the name of, then worked out how to do it in Gimp when that was my go-to image manipulation software, and have since found ImageMagick is better for many tasks.

I'll almost certainly find even better ways to do the tasks I currently do, using new and better tools. But that's a good thing, not a bad thing. I'm still in a better spot right now than I would be if I'd never heard of batch processing.

If there's a moral, it's this:

Don't avoid trying things because you don't think you're capable. Capability not only can be but is learned.

Affinity Publisher editable object styles workaround

Unlike e.g. InDesign, Affinity Publisher doesn't have full object styles. You can create 'styles' as, essentially, presets, but you can't update or edit them and have the effect propagated to the contents of your project, in the way that you can with paragraph or character styles.

Suppose you want to apply, say, a coloured stroke to some subset of the objects in your project to make some cut-out art pop or to distinguish frames. You know that you might later need to tweak the colour, or the width, or the opacity, for all those objects.

An Affinity Publisher 'style' doesn't let you do that.

So here are two easy hacks to capture some of the basic functionality of proper object styles.

1. Names-as-styles

When you apply a style to an object, rename the object to the style name. Then when you need to change the look of objects with that style, use Select Same → Name from the right click or the Select menu.

This works for things like repeated placed images and copy-pasted frames: cases where all the instances will have the same name by default, so you don't need to do anything as you go.

2. Tags-as-styles

But suppose your project requires certain naming conventions. Or you're creating elements in such a way that objects in the same style won't have a uniquely shared name. Then you can get the same results but using layer tags.

Assign a specific unique tag to objects which are meant to have the same style, from the bottom of the right click menu. Then you can later use Select Same → Tag Color to adjust the look of all those layers at once.

Just be careful with nested and grouped layers, where tags get inherited by default.

Updating past TTRPG work for the year 2024

I set aside some time this week to apply last year's fancy purple Periapt Games logo update to most of my previously published work. Along the way I found myself freshening things up with some nicer graphics, improved hyperlinking, typo corrections, and a little bit of extra content.

This meant minor updates to half a dozen different things (note, affiliate links):

120 Fantasy Food Inspirations

We Will Yet Triumph: An Imagination Game

Spells: Made More Magical (5e)

and

Spells: Made More Magical (system neutral)

Almanac of the Archaic

A Fistful of Curiosities: Ten Site-based Mysteries, Oddities, and Puzzles



 

 

 

 

It was a bit more of a process than I expected, with all the files in different layouts and most of them generated using technology (LaTeX) that I stopped using for TTRPG content a year ago. And then there was the extra work of uploading and double-checking the files, resetting the file previews, updating storefront images, etc. It's definitely enough to make me want to stick with my nice new logo for a long time.

If you bought or received any of this older work via DriveThruRPG, you can download the new versions for free through your account there. Thanks!

Hyperspecific bugfix notes in transit from the past: Blogger CSS

While trying to increase line spacing to improve accessibility for this blog, I ran into an interesting series of issues. I'm recording the solution here in case I need it again or on the off-chance that it helps someone else.

For things on the Blogger platform (like this blog), you can choose a theme separately for desktop and mobile, or have the desktop page served to mobile. If you have a custom theme like I do, and you want to have a nice responsive mobile version of the site, then Blogger (behind the scenes) apparently takes the custom theme, runs it through some arcane processes, and spits out a mobile version.

So I encounter the following timeline of problems:

1. The theme is partially customisable in Blogger settings but to make real changes (in this case, to the line height) you have to dive into the CSS/HTML. I took a web design course more than a decade ago, so I am rusty. It takes some time just to find out how to get in there.

2. I notice that I can fix the line height in the CSS using (go figure) line-height, but the change doesn't propagate to mobile. It looks like the .mobile tag doesn't do anything. Say I change the line height to an absurd 100×. It looks absurd on desktop. I simulate a mobile device in a desktop browser by popping "?m=1" at the end of the URL, and it looks absurd there. On an actual mobile device, it's unchanged.

3. I deviate from proper CSS and get lost in an endless maze of <b:if cond='data:mobile'> or possibly <b:if cond='data:blog.isMobile'> or <b:if cond='data:blog.isMobileRequest'>, none of which help.

4. I get frustrated and go poke at the custom fonts instead.

5. It turns out the mobile view ignores custom fonts – again, only on an actual mobile device.

6. At first I think it's a caching problem on my phone, but changing font colours and banners and things works fine.

7. A search turns up this helpful article. It's a web safe font problem. My desktop browser recognises the custom title font I chose on a whim in Blogger settings (Molengo), but my phone browser doesn't. But! I just need to make the CSS load the font and add a line wherever it's needed and then it changes for mobile too.

8. ...It turns out that the google font called 'Molengo' looks completely different to the built-in Blogger font called 'Molengo' which I selected, so my nice font choice is broken on both desktop and browser. Not too big a problem; the google font version is okay and now I know how to make it work on mobile, so I leave it as is.

9. Back to trying to increase the line height.

10. It slowly occurs to me that a 'web safe font' problem is explicable but a 'web safe line spacing' problem is not plicable at all. That means there is an unrelated issue.

11. I google for more answers, get none, and start googling ever more tangentially related search keys and scrolling through increasingly unrelated answers. I try "@media only screen and (max-width: 600px)" instead of .mobile, and again certain test changes take effect on my phone and certain others don't. I can change colours and backgrounds and all sorts of things, but not certain font facets.

12. Finally an answer to a question about I don't know pineapples or something reminds me that !important is a thing. I try it, on a whim. The line spaces change.

13. I eventually work out that something, somewhere, is (re?)setting my line spacing (and a few other things, like font size and letter spacing) on mobile. Either those values are being overwritten by the Blogger post-processing step, or possibly I am an idiot and mobile-only elements buried elsewhere in the stylesheet are taking precedence due to CSS specificity values or something like that. The latter would seem more likely prima facie, except that I searched the CSS for a long, long time and could not find any trace of conflicting elements.

14. Whether it's the one reason or the other, I don't have a better solution than using !important. So I copy the formatting I want from throughout the stylesheet, duplicate it, and slap it inside .mobile tags with !important added on each line. Why go out of my way to do it that ugly way? Because !important is slightly scary and if it breaks something in the future (and I forget about this bugfixing session) then the fact that the blog will break on mobile and not on desktop may make tracking down the problem faster.

So, attention future me: If you need to futz around with this again, start by scrolling to line 554 of the CSS.