triggering key presses in max msp (send key press)
Hi,
what I want to do, is send key press data from max msp. I realise it's easy to recieve key press data, but can you send it back?
For example, I create a patch which sends key press, or game controller data, that could then manipulate another programme. A slider reaches the value 10, and this sends a Q trigger, so that the computer thinks i've physically pressed the key "Q".
If this isn't possible, can you interface midi values to send data via a gameport, since they use the same socket?
Any help appreciated!
Okay, i've found which will enable this using this program:
it should be possible to send midi data from max and convert it to keystrokes using this. which should enable what i want to do here.
i would still be interested to hear if this can be done directly from max though, or any other ways anyone knows?
You could use Masayuki Akamatsu's [aka.keyboard] external or Bill Orcutt's [shell] external in conjunction with the applescript Extra Suites add-on.
lh
Quote: genfu wrote on Sat, 07 February 2009 12:58
----------------------------------------------------
> Hi,
>
> what I want to do, is send key press data from max msp. I realise it's easy to recieve key press data, but can you send it back?
>
> For example, I create a patch which sends key press, or game controller data, that could then manipulate another programme. A slider reaches the value 10, and this sends a Q trigger, so that the computer thinks i've physically pressed the key "Q".
----------------------------------------------------
Not sure exactly how, but via the mxj / java interface in max you should be able to send keystrokes to other apps (or just "to the computer") which would register in other programs (the active one). Or the aka.keyboard can do it, though mac only I think.
Quote: seejayjames wrote on Sat, 07 February 2009 22:39
----------------------------------------------------
> Not sure exactly how, but via the mxj / java interface in max you should be able to send keystrokes to other apps (or just "to the computer") which would register in other programs (the active one). Or the aka.keyboard can do it, though mac only I think.
>
>
----------------------------------------------------
yes i'm on windows unfortunately. so far i've sort of got this working with bome's and autohotkey, where autohotkey allows you to also convert API keystrokes to joystick clicks etc., which bome's doesn't.
i'm trying to control quake parameters via midi but quake only recognises direct input keystrokes in the game (while they register okay on the console). i can use midi to control mouseclicks ie. a midi key hits mousebutton1, which works in the game. but at the moment that means im limited to only a few controls. ideally i need the keystrokes to register in the game, although if i can get joystick buttons to register properly or some other aux inputs, that might be enough. its not possible with bome's or autohotkey to send direct input keystrokes as far as i can tell.
if it were possible to send direct input keystrokes via javascript from max/msp, that would be a much better solution. unfortunately, i've no idea how you would go about this.
i also need to get mididata from max to bome's, which i assume is fairly straightforward. but sending the midi data to my korg nanopad, which is what i'm experimenting with at the moment doesn't appear to work... so perhaps the nanopad doesn't have midithru? in that case i suppose i need to use a virtual midi cable such as midiyoke? is midiyoke the best one to use?
the other long term question is if it's possible to translate forcefeedback directinput information (coming from quake?) which would go back to a forcefeedback joystick, back into midi data, which would mean the game can send information back to max msp.
short video demonstrating FOV automation in quake, controlled via max msp, where a midi slider controls the rate of change.
Here's a simple(ignores modifier keys) mxj that might help. I used something like it to send key events from Max to word on a pc.
just tried it with a couple of games and it worked ok, don't know about quake though..
jan.
Quote: jan.jdo wrote on Sun, 08 February 2009 12:31
----------------------------------------------------
> Here's a simple(ignores modifier keys) mxj that might help. I used something like it to send key events from Max to word on a pc.
>
> just tried it with a couple of games and it worked ok, don't know about quake though..
>
> jan.
----------------------------------------------------
YES! this does exactly what i want, it works perfectly in quake. i can ditch the convoluted midi and autokey stuff and pipe this directly to quake now. thank you so much!
Quote: jan.jdo wrote on Sun, 08 February 2009 13:31
----------------------------------------------------
> Here's a simple(ignores modifier keys) mxj that might help. I used something like it to send key events from Max to word on a pc.
>
> just tried it with a couple of games and it worked ok, don't know about quake though..
>
> jan.
----------------------------------------------------
That is so awesome. I've got it hooked into Unreal Tournament 2004 (yes I'm a bit of a junkie) so that I'm constantly jumping. Not really that "useful" but it's pretty funny. Possibly some way to configure it so some events can be hands-free, with some randomness... don't know if it'll help my score but it's hilarious to watch.
Two questions: any way to send a "keydown" versus a pressed key (with keyup)? So I could move forward, say, continuously rather than in bursts. Also when it's two quick taps it does a "skip-jump" forward rather than just walking forward, which is fine but not always what you want.
Also, can mouse clicks be sent? Would love to have some kind of auto-firing. I guess I could configure keystrokes to fire instead, but am just curious if the robot class in the java can send clicks.
Thanks for the patch and the java file, they're great!
--CJ
I remember having problems with the keypress before -think i just had to send multiple keypress messages, but i could add a separate keyrelease which might help.
the mouse clicks should be no problem though- Robot allows button, position and wheel events. i'll try and add these tonight.
jan.
Quote: seejayjames wrote on Mon, 09 February 2009 00:07
----------------------------------------------------
> Two questions: any way to send a "keydown" versus a pressed key (with keyup)? So I could move forward, say, continuously rather than in bursts. Also when it's two quick taps it does a "skip-jump" forward rather than just walking forward, which is fine but not always what you want.
>
> Also, can mouse clicks be sent? Would love to have some kind of auto-firing. I guess I could configure keystrokes to fire instead, but am just curious if the robot class in the java can send clicks.
>
just added most of the rest of the functionality from the Robot class - mouse buttons, wheel and position.. and separate key press and release. It's a bit rough but works ok.
i dont have Unreal tournament to try but i did manage to get smooth movement in another game so maybe it'll help.
jan.
this is really useful...
any chance you could add joystick buttons?
Sorry - don't know of a way to easily add joystick button control like this.
Have you seen glovePIE? i havn't tried it myself but it might be an option.
jan.
Quote: genfu wrote on Thu, 12 February 2009 07:21
----------------------------------------------------
> this is really useful...
> any chance you could add joystick buttons?
----------------------------------------------------
Quote: jan.jdo wrote on Mon, 09 February 2009 19:14
----------------------------------------------------
> just added most of the rest of the functionality from the Robot class - mouse buttons, wheel and position.. and separate key press and release. It's a bit rough but works ok.
>
----------------------------------------------------
Thanks so much for the efforts. This is really great. I love the idea of experimenting with auto-clicks, dragging, and scrolling... I've already put it to work viewing long websites at whatever scroll-rate I want. Being hands-free as much as possible is fantastic, considering how much I'm on the computer (maybe need to unplug? get some R&R?)
Haven't tried it in Unreal Tournament yet but I'm sure it will work fine. Now the question is... how to use jit.desktop to grab frames, then analyze successive ones for motion (assuming you're standing still and looking for victims), and auto-aim for a kill?? An auto-aim hack using Max, that would be awesome... :)
Regarding genfu's question about joystick buttons, have a look at the [hi] object which one could add along with this, no need to incorporate it into the java. Very straightforward if the [hi] sees the joystick.
Thanks again, this gives me a lot of ideas and helps me understand some more java. Always interesting how straightforward it turns out to be in the end, though I imagine there was some research and trial-and-error getting there... your efforts are appreciated!
--CJ
Quote: seejayjames wrote on Fri, 13 February 2009 12:27
----------------------------------------------------
>
> Now the question is... how to use jit.desktop to grab frames, then analyze successive ones for motion (assuming you're standing still and looking for victims), and auto-aim for a kill?? An auto-aim hack using Max, that would be awesome... :)
>
I've been trying to think of some interesting things to do with this but that sounds excellent:) If you ever do anything like that I'd love to see it..
I misunderstood what was wanted with the joystick -i thought he wanted to simulate a button press like we are doing with the mouse.
Have now added ability to set modifier keys and added some other stuff which doesnt quite work yet -i can send it sometime when it's done if you
jan.
Quote: jan.jdo wrote on Fri, 13 February 2009 18:43
----------------------------------------------------
> Quote: seejayjames wrote on Fri, 13 February 2009 12:27
> ----------------------------------------------------
> I misunderstood what was wanted with the joystick -i thought he wanted to simulate a button press like we are doing with the mouse.
Yes, this is what I was after actually, and I wanted to do it with no joystick actually present (i don't own any joysticks). I can live without it for the time being though.
Here's a couple of short videos of what i've been using this for in Quake:
- Game with FOV, host_framerate (game speed), fog density and fog colour all automated in Fitzquake. Keystrokes are bound to aliases, cycles and increments stored within a .cfg file. A Keystroke is bound to also trigger that config file. Patch reads the game logfile to get info about events such as "pick up ammo" which then trigger bangs to turn these cycles on and off. At the moment in a fairly arbitrary fashion. Midi buttons on my korg nanokontrol can turn the cycles on and off and the sliders adjust the rate at which the cycles occur.
- Same patch as above, this time playing around with noclip. You get some quite interesting effects due to the changing colour of the fog:
- Earlier version of the patch which instead uses jitter to cycle the hue. The problem with using jitter.desktop is it rapidly becomes simply too slow for the game to be playable once you apply a few effects. It's a shame because the original idea was to use jitter to get some nice trippy effects happening in a way which is interactive with the game... however it doesn't seem to be that practical. So instead for the time being i'm focusing on manipulating existing visual effects within the game engine, as you see in the videos above:
Anyway it's a fun little project aided very much by the keystrokes mxj, so thanks for that. The next thing i'll probably add is some audio processing of the game sounds, and perhaps a slightly more structured and subtle use of the game parameter changes.
Interesting idea about getting jitter read the screen and perform some sort of auto aiming. Another idea I had was to read game information by getting jitter to 'see' what is happening, or perhaps use some voice recognition functions to 'hear' if a monster makes a sound and use that to trigger effects. Since discovering the console logfile and getting it to read that, i've found this is a much simpler way of doing things. Perhaps for the auto aim you could use something similar - if it were possible to get the game to print a log file which details co-ordinates of where the player is and where the monsters/opponents are, then you could theoretically create and algorhythm to adjust the aim accordingly based on that?
-->
Anyway it's a fun little project aided very much by the keystrokes mxj, so thanks for that. The next thing i'll probably add is some audio processing of the game sounds, and perhaps a slightly more structured and subtle use of the game parameter changes.
Interesting idea about getting jitter read the screen and perform some sort of auto aiming. Another idea I had was to read game information by getting jitter to 'see' what is happening, or perhaps use some voice recognition functions to 'hear' if a monster makes a sound and use that to trigger effects.
-->
Wondering about the audio processing, have you been able to get the audio into Max directly? This has been an issue for me in the past, where I had to pipe the audio out physically and then route it back in to the adc~ (with a full-duplex sound card only), since with the game or other app running I couldn't seem to get Max to hear it while it was playing. Probably am missing something, I hear about Soundflower on the Mac which apparently can do this, but for Windows?
Also the auto-aim is just a thought. It might be straightforward, or not, but my thought was that assuming you're standing still, you trigger the analysis, and any spot there is a difference between frames the mouse would move towards that (using a kind of successive analysis, but choosing only one area). You'd have to make the analysis, decide where the view would move to, then turn the analysis off briefly (so you don't get the overall scene motion being analyzed too), move a bit, run it again, etc. Probably depending on how far the motion is from the center, your mouse would move ("be moved") a specified amount and in that direction. All this could happen really fast, plus you could maybe have an optional Fire command once you're there. Not at all sure what the results would be though.
It's interesting to think about and would be cool to see it in action. However I like the idea of the psychedelic processing/controlling vid parameters etc. as well as sounds in real-time better, and is possibly easier. With a more customizable game this would be even more so... anyone know of a game with tons of changeable parameters that could be mapped to keystrokes? I was thinking Second Life, but there's a lot of layers there and it's not as much a "up-to-the-second" kind of environment as a shooter or other game seems to be. Some kind of Quake/Unreal Explorer which has that all built in, and ideally without needing to learn too many console commands, though this may be the only way to get to them programmatically.
Interesting about having the MIDI controller too, that would be useful, since I originally had a keystroke to trigger the events at all... but once the game is the active app, those keystrokes go to the game, and my "trigger" key no longer will shut it off. Maybe a special function key would work---as it is, I need to switch apps to Max to stop it manually. So having the midiin or ctlin control of that would bypass that issue (as I don't imagine the game picks this up by default, or at all).
This is a cool thread and the mxj makes all these new ideas possible. Maybe we need to just make our own game in Jitter :)
> Wondering about the audio processing, have you been able to get the audio into Max directly? This has been an issue for me in the past, where I had to pipe the audio out physically and then route it back in to the adc~ (with a full-duplex sound card only), since with the game or other app running I couldn't seem to get Max to hear it while it was playing. Probably am missing something, I hear about Soundflower on the Mac which apparently can do this, but for Windows?
I haven't attempted it yet, and although I dream of a version of Quake which fully supports ReWire, I think the easiest way to do it will be to physically route it out and then back in to my soundcard. I've got enough inputs and outputs for that not to be a problem. I can't see any other way to do it without changing the game itself somehow to route the audio into max/msp, unless there were an object which enabled max/msp to 'hear' the windows desktop (except max/msp itself)... that would actually be incredibly useful. There is a related topic to this issue here (http://www.renoise.com/board/index.php?showtopic=16203&hl=). Total Recorder does work for recording the desktop, but I don't think it would be appropriate for streaming audio. Jack might be worth looking at for this though.
> With a more customizable game this would be even more so... anyone know of a game with tons of changeable parameters that could be mapped to keystrokes?
It's true that I started this thinking i'd use Jitter to process Quake, but now i'm trying to do stuff with the graphics in the game, it would make sense to use a more recent game which has greater graphical capabilities (or else an older game which runs quicker than Quake like Doom). However as far as I know few games offer the options/modability quake does in terms of being able to manipulate game parameters easily, since quake enables the average user to input quake C from the console, which is very straightforward, and you can also write, edit and exec the cfg files. The first game i'd look towards after Quake would probably be Quake 3 or another game based on the Quake engine, or something similar like Unreal.
> Interesting about having the MIDI controller too, that would be useful, since I originally had a keystroke to trigger the events at all... but once the game is the active app, those keystrokes go to the game, and my "trigger" key no longer will shut it off. Maybe a special function key would work---as it is, I need to switch apps to Max to stop it manually. So having the midiin or ctlin control of that would bypass that issue (as I don't imagine the game picks this up by default, or at all).
Another way to do this is assign a key in the game which is bound to exec a .cfg file or print something unique in the usual console log. Have max/msp read that to switch the patch on and off.
> Maybe we need to just make our own game in Jitter :)
Or convince someone to write a fully duplex hack for a game which enables access to all game parameters directly from max/msp without the need to do it via keystrokes... ;D
Hi Stefan
AFAIk the bad version number error just means that the JRE on your computer is not the latest version - i compiled with 1.6.
I've attached my latest attempt and included the source as well - I think it should compile on mac with no changes.
jan.
Quote: stefantiedje wrote on Tue, 17 February 2009 10:18
----------------------------------------------------
> I get an error:
> java.lang.UnsupportedClassVersionError: Bad version number in .class file
>
> Is there a way you could post the source as well? It might be possible
> to adapt it to work on a Mac as well...
>
> Stefan
>
I've been building a patch which uses this autobot mxj object, however I can't get it to work when a save the patch as a collective or application.. the keystrokes only register when the patch is run as a normal .maxpat
Any idea why or how I get it to work in a collective/application?
genfu wrote on Sat, 07 February 2009 19:58Hi,
what I want to do, is send key press data from max msp.
haha, you know, that is exactly what i was thinking
when i saw your quake project.
talking as keyboard to the OS from max will be the
most you can get out of it as a remote for other programs.
-110
any idea why that is? is this general to mxj objects... they won't work in collectives... or?
if included or provided at correct path it should work?
_
the autobot.class object is included, but for some reason it doesn't seem to work in the collective
I made a standalone with something like this in max 4 and it worked ok.
think you need java/lib/max.jar needs included too?
jan.
hmm, don't know, i just tried including that file also but it still doesn't work. I am using Max5, don't know if that should make a difference. Does it make a difference where the autobot.class is located (because it's not in the Cycling '74javaclasses at the moment)? Any ideas?
Just tried this and it worked fine in max5 too:
Built an application
didn't add anything to the build script
When the app was built - went to its support folder and created a java folder inside.
made a lib folder and a classes folder inside the java folder.
copied max.jar to the lib folder and copied autobot.class to the classes folder.
HTH jan
thats excellent, it works now. fantastic, thanks!!
Has anyone kept these Java files? They got lost since the forum was upgraded...
Seconded - I could use these, too.
Copy the ring_keyPress.class file to your max 'java' folder.
C:\Program Files\Cycling '74\Max 6.1\Cycling '74\java\classes
Then you can access it by creating a 'mxj' object:
mxj ring_keyPress
It will instatiate the 'ring_keyPress' java class (see attachment).
You can send a string containing the code of the key to be stroked to the inlet of the 'mxj' object, and voilà.
You can see all key codes here:https://docs.oracle.com/javase/7/docs/api/constant-values.html
Thanks, Reya Gistrout! Now we just need the attached file ;-)
Check again. I have edited the post. =)
There is a .zip file now.
It was mentioned above that it was possible to designate key-down as separate from key-up. How is this done? I would like to hold modifier keys like control or shift while hitting others.
There is some clarification that's in order with this post. Through most of the thread I believe they are actually talking about [mxj autobot]
but towards the end Reya introduces [mxj ring_keyPress]
Can this code trigger shortcut into within another application like FinalCutProX ? If yes, is it because it would send the shortcut send the active application right ?
Just trying to understand how this whole thing work...
(here i just explain what i wanna do https://cycling74.com/forums/yoga-bodywork-using-gyroscopes-while-simple-repetitive-fcpx-editing )
I've been using [mxj autobot] and yes, it simulates keystrokes just as if they came from the keyboard, so whatever application is active will get them.
great, thanks ;-)
http://www.11olsen.de/code/max-objects/11strokes
Oh thanks ;) This 11strokes2.mxo looks really nice too...!