Zum Inhalt der Seite gehen


if I had a lot more time I think I might write a book on my ideas about "adversarial automation".

The idea that the point of computers is to help the humans do their job faster and easier, and sometimes the computer or the software on it is the enemy in that battle.

Als Antwort auf Foone🏳️‍⚧️

because I see a lot of people approaching automation from this attitude of "software/sites should have APIs so that users can write software to automate it!"
and while that's not wrong, exactly, it's also not the attitude I think makes the most sense, you know?

We do not ask for access. We don't need to get permission to be able to automate our tasks.

tante hat dies geteilt.

Als Antwort auf Foone🏳️‍⚧️

There is always API 0: acting like a human/browser/user.

The first API is "fuck you I'm doing it anyway". Any additional API the program provides is merely a helpful shortcut

teilten dies erneut

Als Antwort auf Foone🏳️‍⚧️

You see the point of this a lot in API design, where a company is like "okay we made an API but we limited it a bunch because we are scared about cheaters/bots/scrapers/whatever", while the things they limit are things a user clicking links can do in 2 seconds.
Als Antwort auf Foone🏳️‍⚧️

like, if your API doesn't provide me a follow_user() call, but the user can follow anyone by clicking one link?

Your lack of a follow_user() call is not going to stop me. I'm just going to click the link, automatically.
Having an API 1.0 doesn't mean API 0 goes away.

Als Antwort auf Foone🏳️‍⚧️

And I think this is an under-discussed part of automation because it's associated with spammers and such, but they're only one possible user of this. By making it better known it can get used for more legitimate uses
Als Antwort auf Foone🏳️‍⚧️

The basic philosophy of adversarial automation is that the software/website is the enemy.
Als Antwort auf Foone🏳️‍⚧️

I'm talking less like "you're in a constant arms race with the people maintaining the official API as they try to stop your spamming" and more like "Your lab depends on this program from 1996 and there's no updates and no way to automate it"
Als Antwort auf Foone🏳️‍⚧️

and the answer is really that of course you can automate it. Stick it in a VM, OCR the screens, inject your own DLLs, puppet the keyboard and mouse!
Als Antwort auf Foone🏳️‍⚧️

my point is that every program, every website, DOES expose an API, you just need to know how to best use that API.

That API being "the access they provide for humans"

Als Antwort auf Foone🏳️‍⚧️

For websites this is forms and links. For desktop applications this is buttons and windows and keyboards.
Als Antwort auf Foone🏳️‍⚧️

And I think (in part because it's affiliated with Bad Actors like spammers), a lot of programmers don't consider all their options in these areas.
Als Antwort auf Foone🏳️‍⚧️

And that's really a shame. Computers should be used to automate things. We spend way too much time dealing with shitty sites and shitty programs because we have no choice and think we can't automate them away.
Als Antwort auf Foone🏳️‍⚧️

well, that's wrong. We can absolutely automate them, it just takes a little more work and some different strategies
Als Antwort auf Foone🏳️‍⚧️

I think of this as a short term vs long term thinking sort of problem. Like, a lot of programmers are stuck in the "should" part of thinking about programs and sites.

Yes, the program SHOULD be open source, so you can just fix the UI. Yes, the website SHOULD have an extensive API so you can easily automate it.

Als Antwort auf Foone🏳️‍⚧️

and if you want to automate it today, your only options are to be adversarial about it. It's the enemy, you pretend to be a human user and automate the interactions with the app/site. It's the only way.
Als Antwort auf Foone🏳️‍⚧️

by all means, try to switch to open source alternatives or get them to fix it or add an API.

But at the end of the day that's asking "the enemy" to do something for you, and they are under no obligation to listen to you.

(They may not even exist anymore, given that a lot of the times when I've used this sort of Adversarial Automation it's been focused on software from decades ago)

Als Antwort auf Foone🏳️‍⚧️

It's also a thing that intersects with the way a lot of people online are thinking about computer-use as something they do as a personal hobby, you know? They can run any OS, any software they can legally (or even illegally) install, they can use any options they want
Als Antwort auf Foone🏳️‍⚧️

But the fact is, often times people have jobs where they aren't self-employed and have to work for other people, and those other people can be like "you need to use FooBaz 2007 for this job".

Would it be easier to automate if you were using OpenBaz? Certainly! But your boss can still tell you "no, we're not switching to OpenBaz, we need to use FooBaz 2007"

Als Antwort auf Foone🏳️‍⚧️

And now your options are basically just:
1. Get a different job
2. Use FooBaz 2007 manually
3. Adversarially automate FooBaz 2007
Als Antwort auf Foone🏳️‍⚧️

One example where this came up in my career was when I was working for an educational book creator/publisher. Apple had just added a bookreader tool for iphones/ipads/etc, and we had a lot of colleges asking if we could provide our textbooks in that format.
Als Antwort auf Foone🏳️‍⚧️

well, at the time the only way you could make books for apple devices was to use the book creation program, which was basically a word processor. It was focused around the idea that you would create your books inside it.

Well, we already had our books created.

Als Antwort auf Foone🏳️‍⚧️

We didn't want to have someone retype them.

We could import them as plain text (or DOC, I think?) and that'd get the actual text content of our books with some minor formatting, but we had very interactive and multimedia books. Tons of images, cross links, quizzes, and so on. Pretty much all things that the apple book format supported, but didn't support importing.

Als Antwort auf Foone🏳️‍⚧️

So our options were basically:
1. Hire a bunch of people to painstakingly re-create our books inside the Apple Books tool
2. Adversarial Automation, baby!
Als Antwort auf Foone🏳️‍⚧️

We figured out how much could be imported, and what was left out. We figured out the limitations of the undocumented applescript interface. We figured out we could build complex HTML documents, copy them, and then have the keyboard automation press "cmd-V" and they'd be brought in without issues.
We automated away the bad UI that was going to make it too expensive to publish on apple platforms.
Als Antwort auf Foone🏳️‍⚧️

Should Apple have provided better docs and interfaces and APIs?
Yes, of course!
We asked for them.

But at the end of they day, they may not. And we need to publish this stuff soon, not in several years when Apple decides it might be a good idea for the next revision

Als Antwort auf Foone🏳️‍⚧️

My overall point is something like:

By all means, use APIs and official channels and built-in scripting support if you can.

But remember those are only shortcuts to automation. You can always ignore them and off-road.

Als Antwort auf Foone🏳️‍⚧️

My silliest example of this sort of thing:

I was automatically taking screenshots of a DS game in an emulator. my program would load a savestate, jam some new data into the DS's RAM, hit a button, then screenshot it. But the emulator was showing a "SAVE STATE LOADED!" text overlay over the game's window, no matter what option I set.

Als Antwort auf Foone🏳️‍⚧️

I go on the dev's discord/IRC, talk to them about making it an option, they say they've considered it but it's low priority.
I look into building the software myself, but it's very complicated on windows, with a lot of dependencies and such...
Als Antwort auf Foone🏳️‍⚧️

I open the EXE in a hex editor. Find the string "SAVE STATE LOADED!", and change the first character to a NUL.

Now the emulator is still showing the message, but since it's zero characters long, it's invisible. Problem solved.

Als Antwort auf Foone🏳️‍⚧️

my overall thesis is, I guess, that programmers need to remember they can say "fuck you" to software more often.