shell
Thanks for the swift reply.
Unfortunately @forkmode 1 doesn't change anything here.
Hm, it worked for me on macOS 11.6.3. Maybe Apple has killed vfork (or aliased it to fork) on macOS 12.
Fixing this perfectly will probably require some extreme re-engineering of shell (where its spawned process is basically always running as a server and we pipe commands to it for execution). And while that model is probably a better way to go in general, it's going to involve some work. Can't say when that work will happen, though.
ok, thanks for letting me know. have to look for another solution then.
For the record, tried the current shell (Jul 2021) on mojave and it works fine.
Also quickly checked out the old shell (commit from oct 2018) on github, recompiled for arm64 and found the same problem with the drop outs. Seems definitely related to macOS 12
Hi Jeremy, Can you please explain a bit about the @forkmode?
I asked exactly the same question as Volker on July 20 2021 here, regarding audio dropouts — only difference is I'm using 10.14 Mojave.
Back then you simply replied about "using a different fork" but without giving any detail on what it was or how to use it. And obviously there's nothing about in the help file, I first hear of the attribute today :(
Well now you know! Sorry for the incomplete information last July.
At least up until Big Sur, there are two types of fork available on macOS, fork() and vfork(). vfork() works differently under the hood and is significantly faster.
If and when I do the rewrite alluded to above, it won't matter anymore, since the object will fork at instantiation time and that extra process will just wait around until there's something to do.
I see, thanks !
So just to clarify, @forkmode 0 (default) is fork() and @forkmode 1 is vfork(), correct?
I will make some tests with my Lisp system and hopefully it does the trick for now.
I also have a machine with Big Sur — I don't use much for a while — I'll also give it a try.
Correct -- you can see the labels in the inspector. Thanks for trying it out.
Indeed I completely missed that, thanks Jeremy !
hi,
on windows machine, i m struggling to find the right syntax to delete a file.
Lets say my that is C:/Users/PC/Desktop/mytest.png
I tried commands like : rm, del, but doesnt work, also with ~, using relative or absolute path, without success.
What would be the syntax to delete a file from shell?
thx!
del is ok, but you probably mess the path.
use conformpath native_win boot and route it to message del $1 > shell.
that would also accept ~Desktop/mytest.png
On Windows the folder delimiter is a backslash (\) not a forward slash (/)! As Max sees \ as a special character, you have to double them up. So you'd have to use:
C:\\Users\\PC\\Desktop\\mytest.png
Thanks a lot @Andy ! :)
Hello, I'm actually using the shell object (on windows 10) to manage lifecycles of OpenVibe scenarios.
This is working great except that OpenVibe's python scripting box cannot be initialized, probably because Python is not in the PATH returned by the echo %PATH% command.
I tried the setx /M "%PATH%;C:\Program Files\Python37" command but got a "register access refused" error.
OpenVibe relies on the users installing Python 3.7.8 (exactly) on their own so I can't see another way to proceed here ...
Any advice ?
Thanks in advance,
Joseph
Did you try to use absolute path to python executable ?
like
C:\Program Files\Python37/python (script to execute)
Hi source audio, and thanks for your quick answer
Unfortunately the scripts are loaded by OpenVibe (which is a visual programming environment too) from an inner python plugin, so I can't call the python binary directly. And anyway the python plugin isn't initialized because OpenVibe doesn't find python.
Right now I'm trying to set the PATH temporarily with something like SET PATH="%PATH%;C:/Program Files/Python37" && echo %PATH% && <launch openvibe command>
At this point I'm stuck because I don't succeed to generate a symbol with a quote inside (for PATH="...."), but this is more a max symbol formatting issue I think, although I'm not sure this will work once the issue is solved ...
Still digging. I'll go ask questions on the OpenVibe forum too in case I missed something.
Will keep you posted, any suggestions still welcome
Cheers,
fighting max symbol and message formatting at same time with unknown scripting language is for sure
frustrating, because you can't tell exacltly what is wrong.
Can you try to execute your commands from command prompt to verify the code,
and then try to recreate it in max & shell ?
Yes, it works perfectly fine from the command line interpreter, OpenVibe not finding python is the only error I was encountering when launching from Max.
Meanwhile I figured out I could write "PATH=..." instead of PATH="..."
I was about to report that it still doesn't work but in fact IT DOES :)
It's just that set "PATH=%PATH%;C:/Program Files/Python37" && echo %PATH% doesn't show the python path appended at the end (which is very weird, considering it is actually appended because it works)
Any explanation of this unexpected behaviour would be appreciated, but for now I can move on.
Thanks for the support !
Hello, I'm trying to run this very simple "cat" command and I get nothing back (it does work on the normal terminal).
cat <(head -n4 /Users/arnaubrichs/Desktop/temp.txt) <(echo 'some input text') <(tail -n +5 /Users/arnaubrichs/Desktop/temp.txt) > /Users/arnaubrichs/Desktop/output.txt
I try to generate HMAC signatures with shell.
When I use the Terminal (OSX.14), I get the same result as from https://www.freeformatter.com/hmac-generator.html#before-output
But I don't when using the [shell] object. See this example :
Any ideas?
I run Windows and not Mac so I don't know for certain but as [shell] is running from inside Max rather than from within the OS itself, isn't the signature going to reflect that the request came from a different source? I presume that [shell] will have to put some sort of "wrapper" around your command string so that the OS knows where to send the reply back to; so if your signature depends on the source that generated it, it will see the two sources (OS and [shell]) as two different things. I'm wondering if there is an alternate "third party" app to Terminal that you could run on the Mac as a command line editor to see if that does a similar thing?
Andy, the signature doesn't depend on the source.
But anyway I think I found the reason of this problem: the echo command used by shell doesn't accept the -n option. I need to use /bin/echo -n instead to get the right result!
Hi all,
I have a problem with special characters (I think...) with commands for shell thru "shell" object:
1 - I take a space of disk like this command:
Get-WmiObject Win32_logicaldisk
in Max with shell is like this
powershell -command \"Get-WmiObject Win32_logicaldisk\"
and it's work very well
BUT
2 - when I put more then one disk I need to add a filter like this:
Get-WmiObject Win32_logicaldisk -Filter "DeviceId = 'C:'"
But as you can see there are now simple and double quotes and I don't know to resolve this to send to shell object without the errors. I tried this:
powershell -command \"Get-WmiObject Win32_logicaldisk -Filter \"DeviceId = 'C:'\"\"
but doesn't work
Do you have any idea where is the problem or how it's possible to do it?
thanks a lot
msnf
this is ok:
powershell -command "Get-WmiObject Win32_logicaldisk"
this is ok:
powershell -command "Get-WmiObject Win32_logicaldisk -Filter 'DeviceID = ''C:'''"
keep doublequotas only at start and end of command
'DeviceID = ''C:''' <-- this are all single quotas, 6 in total
thanks Source Audio for your answer,
but be careful , the PS command is this:
Get-WmiObject Win32_logicaldisk -Filter "DeviceId = 'C:'"
and I need to put it inside shell object
for this at first I need to put the firsts quotes for the whole command:
" Get-WmiObject Win32_logicaldisk -Filter "DeviceId = 'C:'" "
and backlashed them:
\" Get-WmiObject Win32_logicaldisk -Filter "DeviceId = 'C:'" \"
but there are still other quotes inside and this is the pb.
In your answer your 'DeviceID = ''C:''' is not correct for the command - it's like this "DeviceId = 'C:'"
you didn't even try it, or ?
YES you are right!
very special but working like a charm ;-)
THANK YOU very much Sound Audio
msnf
Hi,
I have one problem/question with shell:
I would like to work with ftp/sftp commands on Mac/Win os, but is it possible?
When I work with ftp on Win or sftp on mac I entry inside the soft with a new prompt:
ftp>
or
sftp>
and I don't have the output from the shell object.
Is there is anybody with some experience with this? Is it possible or not?
Some suggestions?
MSNF
I'd rather use node.js, which is a portable solution.
Maybe with the help of https://www.npmjs.com/package/basic-ftp
thanks for the idea, but unfortunately we can't do it :(
===============================
I just found the solution for ftp on windows (maybe useful for somebody?):
ftp -n -s:ftpcmd.txt
the -n will suppress the initial login and then the file contents would be: (replace the 127.0.0.1 with your FTP site url)
open 127.0.0.1
user myFTPuser myftppassword
other commands here...
This avoids the user/password on separate lines where the first 2 lines of the file specify the username and password used for authentication.
source: https://serverfault.com/questions/235684/ftp-windows-command-line
I just tested it and it's work very well!
The next step is sftp ....
OK, the previous commands works in Windows and shell, but impossible to do it on Mac...
somebody has any idea how to write ftp commands on Mac?
On windows is like this:
powershell -command \"ftp -in -s:C:\\Users\\XXX\\Documents\\ftpcmd.txt\"
and works very well
but on Mac os x:
ftp -in -s:/Users/XXX/Desktop/ftpcmdget.txt
doesn't work at all...
Somebody knows what's wrong with this?
Hi I am using macOS Monterey 12.0.
The shell object is not working in max, also tried opening max under rosetta, any ideas?
Mac Os 12.6.6 - is working here
what is meaning "not working" ?
Hello,
Is it possible to run a node script using shell on macOS?
I've tried a cd command to the server directory and then node . but it doesn't work.
What exactly doesn't work? Does it find a node binary? There's nothing special about node wrt `shell`, so it should be possible, but I'd need more information.
set shell wd atribute to node absolute path ?
All right. Here's a more detailed explanation: I have an existing node script which starts a Web server (it is not linked to the node.script stuff from Max) which I open in a browser.
So, the node script is in a folder outside the patcher's folder. I want to start the node script with the shell. So I send the command :
cd /Users/xxx/Documents/Programmation/spat_with_web_server_V2 && node .Please note the dot at the end. What doesn't work is that the sever doesn't start. If I try to connect to it in the brower it's not responding.
Isn't beginning the prompt with a "cd" supposed to give an equivalent result to setting the "wd" attribute?
Well, setting wd to my server directory and then sending the command "node ." doesn't function either.
First thing: make sure the command works as-is in the Terminal.
My best guess is that you need to start your server as a background process, otherwise it will be terminated when shell returns. So stick a & at the end and see if that works?
Yes the command works in the terminal. Sadly "node . &" doesn't. "node server.js &" neither.
hi,
how can I format a text in max to launch a shell program in Windows?
@roald, what does 'which node' return?
@mma you need to format the message as in the cmd.exe terminal. In Max, the \ needs to be escaped (as \\), otherwise it's pretty much the same as what would work in cmd.
@jeremy Unfortunately, yesterday this formatting didn't work, but today it works perfectly.
Many thanks!
So, I am trying to work with it on an M1, but the module is brown. Does it not work with these processors?
Sounds like you don't have the most recent version. Back up to the first post, the latest supports Apple Silicon.
Hi all,
I try to start and stop TighVNC on windows 10 with ps commands.
I have two commands
for start:
cd "C:\Program Files\TightVNC\"; .\tvnserver.exe
and for stop:
Stop-Process -Name tvnserver
All is work fine with start command. Only change is like this in the message object for "shell" object:
cd \"C:/Program Files/TightVNC\" && \"./tvnserver.exe\"
and TightVNC is starting - it's great!
but impossible to stop it....!
I send this message to "shel" object:
powershell -command \"Stop-Process -Name tvnserver\"
but nothing is happening :-(
where is the pb? I tried with this variation:
powershell -command \"Stop-Process -Name \"tvnserver\"\"
and this
powershell -command \"Stop-Process -Name 'tvnserver'\"
but TightVNC is still running.
Into PowerShell program the command:
Stop-Process -Name tvnserver
is working well!!!
is there somebody with the solution or idea what to do?
thanks!
did you try 'pkill'?
no, here are two commands I found:
top-Process -Name tvnserver
top-taskkill /IM tvnserver /
to use directly in Powershell, but only the first one is working.
I didn't find "pkill" command special for powershell, only for Linux and I didn't find how to use it with PS or shell in Max... :-(
don't understund where is the pb with the command which is working inside Powershell but not into shell maxmsp
I think the misunderstanding is that the shell object isn't a terminal, it spawns a single shell which is the only thing running inside of it. So when you run cd \"C:/Program Files/TightVNC\" && \"./tvnserver.exe\", `tvnserver.exe` is running and in the foreground. You cannot interact with it further -- it doesn't have an input pipe to accept commands, like ssh or similar. So if you want to make it stop, send the message pkill to the shell object -- that's a Max message which tells the object to kill off the shell process and whatever is running in it.
I suppose in theory you could try to start the server in the background -- this isn't so straightforward on Windows, though: you can try some stuff from this SO, maybe. And then possibly your powershell command would work.
But no guarantees, this stuff is fiddly and not always easy to manage, especially on Windows.
you can start tvnserver by sending it's path to shell
for example :
"c:/Program Files/TightVNC/tvnserver.exe"
to kill it send :
pkill, taskkill /F /IM tvnserver.exe
to shell
WOW!
it's working great and immediately!!!
thank you very much Source AUDIO
Hi, is there possibility to write shell command like administrator?
thanks
yes, it is possible.
which OS ?
post also what command you want to use, it might make a difference.
OS: Windows 10
commands: "net start/stop tvnserver/AnyDesk"
why the difference?
because on mac OS you need to call sudo and enter password.
On windows, one needs to elevate script.
there are options like creating a script, adding elevated exec rights, then calling that from max.
or bat to exe with admin rights set.
that is ok if you just need that one command.
You can also use elevate.exe for that, there are several compiled binaries,
for example
you pass your command to it.
then there are some powershell commands, like
powershell
start-process PowerShell -verb runas
I am not really daily windows user, and can't tell you what is easiest ...
Great and thank you very much SOURCE AUDIO!
I will try all this. For the moment I found the solutions with the .bat files... but there are some others pbs ;-)
Hi there!
Shell has not been working on both Intel and arm on macOS, for 2 days.
Does anyone know why?
(Shell's latest version installed)
"doesn't work" ?
Nothing changed about shell recently, so what changed on your system 2 days ago?
No, my friend, no os or Max update.
shell: could not load due to incorrect architecture
This is the message in Max's console, but it worked until two days ago!
Obviously I cannot know what is different on your computer. But my wild guess is that something changed about Max's search path and it's now finding an old version of the object which is not Apple Silicon compatible, or you added a package which contained an old version of the object, or something similar. Maybe go through your search path with something like EasyFind and see if you can rule out a second copy of the object somewhere.
Hi Jeremy,
You were right! It turned out to be an issue with the search path. I found an old version of the object in the search path, and after removing it, everything worked perfectly again.
Thank you so much for your availability and support—it helped me resolve the issue quickly!
Hi Jeremy,
We've been in touch before about my package MOZ'Lib, where I use shell to run Sbcl (Lisp) processes on the fly, controlled by bach-based Max patches.
I realized that in certain situations, when triggering some heavy Lisp code (incl. stuck in an infinite loop for instance), somehow the pkill message is not effective anymore. And since I might be running a few objects like these in parallel, I end up with a few sbcl processes visible in Activity monitor, each hogging 99% processor, which I can only force quit from there.
I tried using freebang -> pkill to shell to force quit on closing the abstraction, no effect. In those cases in fact, banging manually on pkill doesn't work either. The only way I found is "killall sbcl", but that's not really an option since other "unstuck" processes might be happening in parallel.
It really comes and goes, some quite heavy Lisp processes have no issue at all, some others the shell objects just seems to lose sight of after a while… In that case, a same instance of shell object can even generate several instances of sbcl one after the other, and if they all evaluate faulty code, they just all keep hanging until I manually kill them in Activity monitor.
(for the screenshot I only ran a single instance, all is coming from the same shell object)
Do you see any reason why this would happen?
Any suggestion to fix it by any chance?
Thanks in advance !
Julien
I do not see any reason why that would happen. I wonder if the sbcl process is forking itself under some circumstances (so that shell no longer has its PID for purposes of kill), or otherwise spawning processes which are outside of the purview of shell?
Since you're able to send other messages to shell (e.g. killall), it seems like the object itself is working. In this situation, if you go to the Terminal and specifically kill the PID of the runaway process, does it go away? Back in shell, if you kill <PID-in-ActivityViewer> , does it die?
I should probably add a way to get the PID of the last-running activity from shell to help debug these situations...
Hi Jeremy
I just tried and yes, if I get the PIDs manually from Activity Monitor, I can kill all the sbcl processes that were left behind (even from a different shell object than the one that started it obviously).
Yes if you added a way to specifically target the last PID for a given shell object, I guess that would definitely do the trick.
But then how to do it specifically, adding some behavior from the last outlet? I guess some people wouldn't be so happy about that ;(
Perhaps something like "shell @name #0_name" + "receive #0_name" ) mechanics would be safer?
Thanks again !
Julien
Hi again Jeremy
Actually the last reply gave me an idea to try. I had no idea it was possible but a SBCL process is somehow able to retrieve its own PID.
So I made an experiment where I would write it to temp file every time a process start, even before the actual (potentially faulty) computation starts.
Two conclusions :
1) I can use that already to force kill the process better. No more issue of sbcl duplicating, it justs stops as soon as a bang the subpatch… And since I write the temp file with a #0_name, multiple instances of my object remain independent.
So that's a temporary fix for my issue, even though I'm not completely happy of writing those files everytime I call sbcl (which happens… a lot, as fast as every 25-50 ms).
2) That solution gives an answer to your previous question :
the PID doesn't change when the difficult part of computation starts. In fact I now print it to file as soon as the process starts, so I guess it is shell somehow that loses track of it for another reason?
Best,
Julien
why write to file ?
add IDs to coll and send out last to kill it or all or ... whichever you need.
@Julien, great to hear, and I'm glad that you have a workaround. I would like to understand why this isn't working, however.
My #1 best guess is that the call to sbcl causes a secondary process to be forked, or you're calling into a new console which then calls sbcl or something -- I'd need to see the message you're passing to shell for that. In any case, that it's some setup where shell can't track the ultimate PID of the process you'll need to kill, due to indirection. There's not much I can do about that, if that's the case, but if you don't mind sharing some of your code with me (privately if you prefer, at my first name + @cycling74.com), that would help me debug this (esp in the case where my #1 best guess turns out to be wrong).
In the meantime, sounds like you can solve the acute problem with this strategy, so that takes some of the pressure off. :-)
@ Jeremy
Typically I send this kind of message to the shell :
( "~/Documents/Max 8/Packages/MOZLib/sbcl/sbcl" --core "~/Documents/Max 8/Packages/MOZLib/sbcl/moz-complete.core" --script /tmp/12136_tmp-in.lisp | /usr/bin/sed "s/^/stdout /" ) 2>&1
What I completely forgot is the sed part, which allows me to reroute the stderror. Do you think that's why?
(if so, my pkill was broken for a couple years, and I only noticed it, which is… good sign I guess? 😆)
@Source Audio
I think you misunderstood the problem. I have potentially multiple instances of the same, shell-based abstraction, to do isolated Lisp-processes in parallel. If I use killall sbcl, I lose everything, including the sbcl instances that are not stuck.
The PID number required to kill the process is given only when that process is started. You cannot decide which PID, it's given by the OS (that's the OS's job).
When the process is started (and sometimes gets stuck) there is no way to get back the PID from the shell object, it's also hanging, unable to interact.
That's why I need each instance of SBCL, when they start running, to store their own PID somewhere. So when I need to kill the process started by abstraction A, I just read that instance's file, get the PID, and kill, without messing up with abstraction B or C. Makes sense?
Hello!
Why did this stop working?
"C:\Users\This PC\Desktop\App Location/App.exe" -f "F:\Samples\Sample_Name.wav"
I'm sure it worked as it should before.
Can anyone explain?
I don't think it stopped working. Maybe you stopped using all backslashes? :)
@Julien
I rather think you missunderstand what I am saying,
instead of writting that PIDs into a file
you can have it as a value
in the patch itself,
in a message, coll, int
anything that can store it and recall it when you need.
@SOURCE AUDIO
But where do you think the PID is coming from?
Who is going to tell me the PID of that process is 48460?
The shell object doesn't provide it. Only the OS knows.
As I said the PID is not a predefined number. You cannot start the shell with a message "sbcl please start on pid no. 859" it doesn't work that way.
You need to obtain the info somehow when the process is already started.
I've done some reading about this problem, and I think I understand the underlying issues. When I have a sec, I'll post a test version of shell which will hopefully be more thorough, less merciful, in its killing habits.
I thought sbcl which as you say is able to retrieve it's own pid
should then be able to report it to shell output.
by the way you can querry named processes easily
no need to run activity monitor.
for example :
top -l 0 | grep sbcl
that will output all sbcl stats infinitelly.
@ JEREMY
Thank you for looking into it, that would be fantastic. Happy to test it whenever you need.
@ SOURCE AUDIO
The shell first outlet is off limit in my case. As I mentioned already, I reroute the stderror of the shell so I can use it to retrieve both the stdout (normal prints) and errors.
The shell outlet is used only for monitoring what the Lisp does in real-time, i.e. everything that is going through a (print x) or similar in my Lisp code, is eventually printed in the Max window. It's all coming from sbcl in an asynchronous way so it's an real mess to handle those different types of print and errors well. Took me 10 years to get it stable and I cannot afford to introduce non-related items like the PID in that pipe.
On the other hand, the actual results of my Lisp calls are all coming through temp files. The data can be huge nested lists (like music scores), so it's actually MUCH safer to write to file (when the Lisp is done), and open as llll with bach.read.
If it came back through the stdout via the shell outlet, my lists would be limited in size and depth so they would be unusable.
Since I'm doing it anyway, writing another tiny file with the PID on every call is really not such a bad deal. And I read it only in Max when I need to force kill.