AI that learns from RAM changes

http://www.youtube.com/watch?v=xOCurBYI_gY

Think Markov Chains building from the temporal RAM states. NES (and robot microcontollers too!) have a little memory, so you can pattern-match it directly.

And you can apply the idea to self diagnose various software! I.e. some unusual patterns would be a sign of errors.

http://www.cs.cmu.edu/~tom7/mario/mario.pdf

>>2
Search that .pdf for "on a scale from" and cringe.

The problem is that he bruteforces the search. Hidden Markov model could have been used instead of emulation. I guess evaluating a markov chain is much more efficient than running an emulator. Moreover, markov chain can easily span minutes, while emulator evaluates just a few frames forward.

>>4
Improving ey's work would be a fun /prog/ community project.

I really like this. Reminds me of some fond memories of early saltybet

>>5

ey

*xir

"Harry Q. Bovik" appears to be some CMU insider meme:
http://www.cs.cmu.edu/afs/cs/usr/bovik/www/

>>8
They are called in-jokes not ``memes''.

The world and rest of the internet is not an imagereddit.

>>9

tell that to Richard Dawkins

>>5
Okay. Here is a simple algorithm, which could be used to replace or speedup emulator (you use markov chains to predict possible good futures, then test them on real emulator). If 100% coverage sampling isn't possible, then we can sample RAM bytes as probabilities of bit being 1 or 0. I.e. 255 is 1.0 probability and 0 is 0.0 probability. That way we can efficiently map gigabyte-sized RAM image into a few kilobyte sampling window.


void markov_chain_ai(video_signal, audio_signal) {
bit_array state;
bit_array predicted_state;

sensors.output_into(state);
user_input.output_into(state);

marko_chain chains[N];

for(chain in chains) {
chain.sample_from(state);
chain.predict_into(predicted_state);
}

for(example in examples) {
sensors.input_from(example.input);
user_input.input_from(example.input);
for(chain in chains) {
chain.train(example.result);
}
}

while(state.are_we_alive()) {
sensors.input_from(emulator.output);
uint8_t best_user_input = 0;
float best_score = 0.0
for (possible_input in range(0,256)) {
user_input.input_from(possible_input);
for(chain in chains) {
// predict_next_state puts votes into predicted_state
chain.predict_next_state();
}
// finally decides which bit is 1 in 0, based on the number of votes from chains
predicted_state.integrate_chain_outputs();
if (predicted_state.score() > best_score) {
best_user_input = possible_input;
best_score = predicted_sate.score();
}
}
state.eval_with(best_input);
}
}


>>11

Oh, and we will also have to keep a few previous states to make sampling temporally aware. Otherwise our emulator-emulator won't be able to predict movement vectors.

>>11
What programming language is this?

>>13
I don't know, but trying to read it gives me a headache. Anything with { and } does that to me.

>>13
>>14
Looks like C.

>>15
That's what I thought so too, but it does have what appears to be C++ features (member function calls with no explicit self-reference), as well as range-based for loops (which do exist in C++, but use a different syntax than what's shown here).

I thought this was how antivirus heuristics work.

>>17
I'm surprised you can think at all.

>>16
I'm confused as well. loops like that exist in Python and Javashit but it's clearly neither of those languages. I'd guess it's a Javashit dialect or something that compiles down to Javashit but then you have uint8_t which wouldn't make sense in such a high-level language. maybe it's just pseudocode

>>17
antivirus does not care about the whole RAM, it cares about IPC (especially starting a thread in another process, which still can be done on windows using undocumented functions), disk I/O (especially modification of executables but with the proliferation of ransomware/cryptoviruses there's also mass modification of documents in the background), keystroke logging etc.

because most of the stuff done by viruses boils down to syscalls, AV heuristics are mostly about setting up a sandbox and emulating those syscalls. the algorithm in this thread wouldn't even understand syscalls - it wants 8-bit sequences in RAM to represent higher values, it does I/O with controller interrupts and it learns to play games this way. this of course falls apart with more complex games where there are multiple variables and progressing in the game does not correspond to them going up (even a simple 2D map confuses it). you can't detect malware this way becuase malware doesn't require variables to go up or down in predictable patterns (with a small exception: malware that abuses kernel exploits will usually seek to modify its creds to higher-privileged ones).

IHBT