11ffload~ external
I just wrote an external using libav that requires the shared ffmpeg dll's. I noticed it keeps working even after removing the additional libs from the search path. So i did a search and it turned out the viddll package comes with this files.
Does that mean we can write cool ffmpeg externals without having to provide the shared libraries.
Is it the same on the Mac?
possibly on windows, but the Mac libraries have to be loaded from the viddll.engine library so I don't think that will work.
I'm seeing dylib files with the same names. isn't it the same prinziple on Mac (xcode), linking the project with dynam. libraries and that's it? I did compile externals for Mac but I think I never had to deal with dylib files. I'm going to find that out
well, def let us know. maybe it works fine!
Some progress here:
On Windows it works fine. It's all good if the ffmpeg dlls are somewhere in the search path, which is granted if the VIDDLL 1.1 Package is present.
On MacOs, after I successfully installed ffmpeg-devel package with MacPorts to get the header files, i'm adding the dylib files to the xcode project and the build succeeds.
But when external gets loaded the dylibs can't be found :
(for some reason it only reports the error for libavformat. I'm also using libavcodec and libavutil.
I don't know if that means the other 2 dylibs have been opened successfully, I assume not)
11ffload~: unable to load object bundle executable
2017-09-20 15:59:52.917 Max[727:707] Error loading /Users/admin/Desktop/max-sdk-7.1.0/externals/11ffload~.mxo/Contents/MacOS/11ffload~: dlopen(/Users/admin/Desktop/max-sdk-7.1.0/externals/11ffload~.mxo/Contents/MacOS/11ffload~, 262): Library not loaded:
@loader_path/libavformat.57.dylib
Referenced from: /Users/admin/Desktop/max-sdk-7.1.0/externals/11ffload~.mxo/Contents/MacOS/11ffload~
Reason: image not found
Do you know where variable @loader_path is looked up or can be set to be the viddll support folder. It seems to look inside the mxo bundle for the dylibs. Do I maybe have to use dlopen() manually in the external? Would I even find the dylibs without hardcoded path? It seems the max.app folder is not in the search path opposed to Max 7 folder on Windows.
I really prefer a solution that refers to VIDDLLs dylibs. Don't want to bundle 30mb with an extenal, 30mb that are already present in every users Packages folder.
I edited the binary in the mxo bundle with install_name_tool. I replace the "@loader_path" parts with absolute paths to "/Applications/Max.app/Contents/Resources/C74/packages/VIDDLL/support/libname.dylib" and IT WORKS. really, could it be so simple? (after reading endless posts, saying you can't link to a dylib outside the app bundle i'm confused)
Is it correct that the Max app path is a fixed thing on Mac and I can rely on this hardcoded path working on every Mac + Max?
Having all applications in the Application folder is something I never liked in OSX. So on my Macs, Max is not in the Application folder, nor in ~/Applications.
And if you create standalones with you external, I guess it won't find the library.
Hmm ok thanks for the info. So the path is not reliable.
You are right, creating a standalone is another problem where it makes no sense to link to Max app.
For now I was hoping I can make it work crossplatform in Max4Live. At least I can be sure the Max app is somewhere on disk in this environment. But i'm kinda loosing hope it will work on Mac.
If I pack the libs with mxo bundle i got a 33mb external. Is that a problem at all?
O.
Omg, turns out every mxo knows the location of the Max binary at runtime and can navigate relative to this location. i'm setting the pathes like this:
"@executable_path/../Resources/C74/packages/VIDDLL/support/libavcodec.dylib"
i didn't noticed first but that's how they find the jitter and audio framework:
@executable_path/../Frameworks/MaxAudioAPI.framework/Versions/A/MaxAudioAPI
@executable_path/../Frameworks/JitterAPI.framework/Versions/A/JitterAPI
Any news on your experiments?
I'm using ffmpeg via shell right now, looking for a more convenient way to reas and re-encode a stream directly in max. Since I'm here, another question, is there a way in max/jitter to play an open video file?(I mean a videofile that is still been encoded). I can open a ts file with jit movie it just lets me watch what has been recorded at the time i ve opened the file. I need to reopen the file everytime I want to so see a new part of the stream.
cheers
I have a working prototype that decodes any file into a buffer~ like the "replace" message of buffer. I have not investigated into the video part of ffmpeg but that is probably what viddll and related jit externals do.
I got a little bit lost and frustrated trying to implement a clever solution for handling big files, i mean files that exceeds buffers maximum size.
Once i can finalize the last 5% of the external i'm happy to share it + source code. But, like i wrote above, it uses a thread to decode an audio stream to buffer~, no video involved.
Basically I think you can put any ffmpeg feature into in external. What helped a lot was to code the function in an executeable to debug and create a working example before moving it to an external.
Even this first step was a lot of struggle as most examples on the web are outdated if you use latest libs.
seems really interesting. I'm looking forward to it, have fun and a nice week ;)
this requires Max 7.3.4
[beta download removed]
scr included. If this works for everybody I'll put this to my site as a download.
And HappyHalloween for all you guys ;)
very cool 11OLSEN, thanks much for sharing and congratulations on getting those libraries working for you!
Hi, 11Olsen. Great external. It's particularly useful when simultaneously dealing with audio and video/3D animation. I've done it in the past and there's some pauses in Jitter while audio is being read from disk. It really should be the standard behaviour of the loading process of buffer~ and other similar objects.
Yes, I want to do an audiofile-browser-like thing. And a seemless prelistening function of files is not possible with the blocking behaviour of the default objects.
I removed the beta version download. I fixed many bugs since then, some of them are critical.
Second, i noticed that when the osx external is frozen into a m4l device the "unable to load object bundle executable" - error is back. The frozen device state seems to isolate the externals from the max app ? Seems this is not finished yet. I have to dive into it again.
As long as the m4l-Editor is not opened, the @executable_path for externals links to the Live app.
Where it finds the Max frameworks but of course not the viddll package. Ok didn't know that.
Maybe that changes with Live 10. For now I see no other way than using @loader_path and including the dylibs in the mxo bundle. sad
Here's version 1.0: http://www.11olsen.de/code/max-objects/11ffload
All ffmpeg dependencies are included for now.
what happened to this external?
can i test it ? ;)
Hi Klaus,
my plan was to put it back online when i managed to recompile for Apple Silicon.
I think that external is really useful especially because it supports many codecs. I keep seeing posts from people who would like to load audio tracks from movie files. It can do that with no problem.
But I'm struggling to create a Mac version which supports intel and arm64. I'm on intel Catalina and after installing ffmpeg with homebrew, I find the files that I need:
libavcodec.dylib
libavformat.dylib
libavutil.dylib
libswresample.dylib
I can use install_name_tool to put these into the external after compiling. Now my idea is to get the arm64 versions of these files and maybe use lipo tool to join them together. But i can not even try without having access to these versions and homebrew doesn't let me install a package of another arch.
Maybe someone using an apple silicon maschine can install ffmpeg and send files to me. Or someone has another idea. I'm trying to avoid having to universally compile ffmpeg myself it seems hard to find the files for arm64 online.
Max ships with "Fat" versions of these libs, maybe you can make use of those?
Nice idea! I'm gonna try it. Also I found something on stackoverflow how to install a second arm-homebrew and make it install arm bottles. I'll report back.
Thanks again @Rob. Only thing i had to do was to install homebrew ffmpeg4 instead of 5 to have the correct header files.
Would someone with an Apple Silicon machine be so kind as to confirm that it works before I make it an official download again?
i get:
11ffload~: cannot be loaded due to system security policy
i am not sure how to get around this. whith my externals i at least get a dialog about it.
Hi Klaus, thanks for testing. Here is a second take on it. This time I had Xcode automatically copy the dylibs to the bundle. If this also doesn't work it's probably a signing problem. Can you try again, please?
sorry, no good news.
i still get this same error.
even when running xattr -d com.apple.quarantine 11ffload~.mxo
the error remains.
with 11strokes eg. i get this dialog, which then solves the issue:
hang on, i missed the NEW link...
yes working now !!
including the dialog from above.
Cool. That makes me happy.
The buffer~ size limit seems to be obsolete. I can load complete movie 5.1 audio stream into a buffer.
I fixed another bug with some avi formats and have revived the download.
I've invested a lot of time for another update. I hope it was worth it.
added importreplace, import and info messages
added resample, metadatadict and import-related attributes
added "channel route" argument for all messages
added ability to import from another buffer~
updated help patcher
Since it has become quite complex now, I may have overlooked one or the other bug. Don't hesitate to contact me. DOWNLOAD HERE
very nice!! been trying to make this work with Native Instruments Stems format which is an .mp4 container of 5 stereo tracks, but it only seems to recognize as 2 channels.. curious if anyone has been able to unpack the stems in Max?
OK, I see the problem. I need to put in a way to set the number of stream to use from the file instead of autoselecting it.
I updated the external, redownload and overwrite. In the new help file check out the "streams" tab. At least all stem files I have here for testing are separated correctly to 5 stereo buffers~ now.
yes!!! you just breathed a whole new life in the Stems format!! thank you!
I noticed for some mp3 files loaded with the external that some milliseconds are missing at the beginning.
All other ways to load the file (Max, Live, external editor) had this additional milliseconds so I knew my decoder is doing something wrong.
I couldn't find out what's wrong by scanning my code. But when I try the ffmpeg cli, I noticed there is something like a start offset detected.
So I'm currently trying to find out how they are getting this value in the ffmpeg executables. Because in my code all timestamps report 0. for the first packet or audio stream start point of the same file.
I don't know how you interact with ffmpeg in your code,
start offsets can be controlled in ffmpeg using -copyts and -start_at_zero
that offset 0.025056 sec is probably to align audio and video stream.
I interact with it in C with the ffmpeg libs, so on a relatively deep level, not by remotely starting ffmpeg executables with command line parameters. I found a lot of files that come with the start offset, so I can't just blame it on the fact that it's a very rare file format and live with it.
I now successfully detect the offset if it is present in the external.
BUT it's not clear how to deal with it as Max and Live for example use different methods.
The external is not involved in this comparison. The file loaded with a replace message to buffer~. Zoomed in to the beginning of the file using a waveform peak as a marker at 454 ms
Loaded into a Simpler or to an audio clip in Live, the action starts about 26 ms later
same in an external editor, which uses lame as encoder
When I look at the reported length for the file, there are more differences.
Max buffer~ and the external editor report 3:15:500. Loaded in Live it says 3:15:578. Only Live devs may know where the extra ms are coming from but a correct start point is more important to have compatible onset times for example.
It kind of forces me to implement a switch or make a decission for compatibillity with either Max or Live.
My observation is that in this kind of mp3 files with offset, only the first packet which has a timestamp of 0.0 is incorrect. All following packets seem to have a time stamp that is already compensated with the start offset.
So normally a rule like "if packetTime == 0. and startOffset != 0. then (packetTime = startOffset)"
would be enough. And Max buffer~ seems to encode it like this. Another way is to simply add the startOffset to every packet timestamp which would produce the same result as Live loading the file.
from https://lame.sourceforge.io/tech-FAQ.txt
===============================================
1. Why is a decoded MP3 longer than the original .wav file?
Because LAME (and all other MDCT based encoders) add padding to the
beginning and end of each song. For an explination of why,
see the questions below.
LAME embeds the amount of padding in the ancillary data of the
first frame of the MP3 file. (LAME INFO tag). The LAME decoder
will use this information to remove the leading padding of an MP3 file.
Modifications to the decoder so that it will also remove the
trailing padding have not yet been made.
===============================================
depending on converter used output could be different.
Lame vs FFMPEG libs, which max uses when importing mp3 into buffer
I went with Live compatibillity after I noticed that Live loads the mp3 with the same offset on Win and Mac. Max on the other hand is not even using the same method on both platforms.
On Mac, the first onset is at 430ms, on Win it's at 454ms, in Live it's at 480ms on both platforms.
You see the distance between the values is about 25ms. All 3 case use a different approach to handle the offset when decoding.
Anyways, regarding the external I have my solution for now and will upload an updated version after more testing.
Maybe I write to Support and let them know about this. I know that MP3 doesn't have much value in audio production, but as long as we offer loading of this format, the results must have consistent timing.