Max and Large Language Models / Generative AI / GPT / etc....
I've been sufficiently impressed by the programming abilities of ChatGPT, and have started to think more about the possibilities of Max assisted programming using large language models like GPT, especially since Max patches are render-able in text form.
Has anyone explored this space and its possibilities, so I don't start from scratch?
Please don’t take this wrong-
I am a classical composer, I prefer analog over digital, but I’m also heavy into certain aspects of technology.
However, personally I will labor until my eyes bleed and make horrendous garbage before ever letting AI / GPT near my work (art).
key word is art.
AI will never make art.
Because AI will never be able to simulate the subtle inconsistencies in a piece of wood that was used to make a Stradivari violin.
Moreover, and I’m just up late writing this so future generations might read it- the one thing that humans can do that computers cannot do is forget. Forgetting is our advantage. Contrary to the idea that you must study history or know everything about something is the idea that (in art at least) makeing mistakes , making ugly, not having any idea what you are doing, remaking the same thing over and over with a different approach, is where you will find the golden nuggets .
this A I G PT thing will be the worst thing that ever happens to art.
I welcome debate on this matter.
-wil pertz
I think it is an interesting idea if GPT could generate Max patches. If it knew the file syntax I guess it could, but max patches are encrypted, or at least Cycling 74 does not make it easy to figure out the file format.
A more interesting route to explore is to use GPT as a tool for computational or algorithmic composition. The easiest path would be to have it generate JavaScript code that you then run in max. Or use it to explore data science to create new tuning tables or representing harmonic spaces as some novel topology.
Or better yet have it help me build a model of my own musical pieces using an RNN. I could then use the model to generate hundreds of pieces in the style of what I have composed.
i think one must understand that for python, java, C++ or html there is a myriad of documented code examples outside which can be used by AI systems to analyze and understand, which is a requirement to translate english to a code result.
for max there is maybe a few thousand sources from two dozen websites or books and whole topics might be covered nowhere.
now the question is if you could get better results using shared databases right in max, where only users and contributors share their data or where data is more or less pretrained and preprogrammed.? that is basically how AI based photoshop plug-ins work for some 5 years.
but i am not even convinced about that. many max abstractions and apps are not documented at all, are full of bugs and errors, others are compressed, or they are copyrighted or noly half-done, so people would just stop sharing them.
i dont want to cause you trouble with my errors and i dont want you to copy my app, either.
and the question after "right" and "wrong" is not at all comparable to say, html and css, which are almost like arithmetics with an officially doumented ruleset what can be done and how, and which is much simpler than those for reproducing knowledge in english or for image generation.
the first attempts to ask the chatbot as is it (public services) for max and pd patches lead to ascii art instead of max4 code or json.
Thanks to everyone for your comments. As Roman points out, initial attempts with ChatGPT were at first quite promising, but the pasted code yields nothing in Max:
Still, I view it as impressive that it actually tried and generated what *looked* like it might work, and that its descriptions of the patches at least are correct... (including using the tilde objects for audio signals).
I did some experiments with the 3.5 chatGPT version and max 8, gen/jit.gen, js/javascript. And although interesting, for the sake of trying to understand the strengths and limitations of the GPT model, the results have been mostly useless :-).
I do think there are some use-cases where you could use gpt to generate some, c-like code for running inside a codebox object in a gen patch. The code i've been getting is close to being functional, but as it stands, in 3.5, you have to babysit it to the point where it's mostly been a waste of time.
You often have to explain, or document basic functions like atanh (inverse hyperbolic tangent), and it's understanding of the overall syntax, within the max environment, is quite limited. But it might help you to get started, if you're wanting to implement well known, and well documented algorithms, like "QR decomposition", ie. machine-learning stuff.
At one point I tried to make it simplify some gen code, by copying the c syntax, but it was quite clear that it didn't understand the basic programming logic at all. It removed a bunch of critical functions, and variables, and in some cases ended up utilizing more multiplications/divisions and function calls, while still breaking the code. I would then ask it to compare the performance of the suggested "simplified" version, versus the original, by counting the critical functions, mul/div etc, and it would flat out state, that my original code was more efficient compared to it's own version, defeating the whole point of the task, which I found kinda funny.
But who knows, perhaps the 4.0 and newer versions will improve to the point where it's usable as a coding co-pilot for certain projects. I can definately recommend trying it out, since it's free, and a good amount of fun.
coding custom openAI models (and training them with custom material later) would be the final victory, but coding your own models is unfortunately quite difficult (i dont even understand the documentation) an and i also not so sure if it make much sense to biuld a custom system with less than a myriad of inputs.
@Gussi - Cool to see that it made some decent progress on gen programming at least (the way I got interested in this topic overall is that I had it generate some Javascript for use within one of my patches, and with a minimal amount of tweaking it worked great).
@Roman - There are tools beginning to emerge that allow for layering additional data sets on top of the foundational models. I won't pretend to understand the process either, but they're coming! (this article from just a few days ago, June 19): https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/use-your-data
the day when an AI is able to code and invent the max patches i am going to code and invent better than i could will also be the day when AI can code the max application better than cycling 74.
furthermore, the AI will then also be able to use AI to create the tools to create max patches, which i dont need anymore anyway, better than me.
which does not matter much, because when everybody else is now able to create himself what i was just about to create, i dont not need to create it anymore.
so why invest much time into this?
"the day when an AI is able to code and invent the max patches i am going to code and invent better than i could will also be the day when AI can code the max application better than cycling 74."
At the rate things are going now that could be as soon as 6 months. I'm absolutely in awe of all the recent developments and how fast all of this is going. One thing that struck me recently was how AI is the first technology we invented that actually takes agency/power away from humanity. The industrial age made our body redundant and the age of Ai will make our brain redundant. It's ok though, i for one embrace our place in completing the final step in the evolution of nature.
Resistance is futile...
Hello Dan, yes I have had this vision and I would like to see it come true. I think Max is the place for this to happen. It is already happening with some of the IRCAM externals.
I have seen ChatGPT being used to create reascripts for Reaper. You ask GPT to write Reaper Scripts, install them in Reaper, and they modify the operation of that DAW. It was a very basic experiment, that could be improved on greatly.
I have a good idea of how I would like AI to help me. I want a modular media programming environment that automatically scripts itself based on my vocal inputs, gestures, and so on. The future vision I see - The end of software. I personally do not like software very much. I like what it does but software can be fiddly and annoying. Also, I use multiple DAWs for my job. I carry out the exact same functions in PT, Live, Logic, Reaper and so on. These basic functions of recording, light editing, basic mixing and so on are always and invariably the exact same but I have to learn the different menus and buttons and shortcuts for each DAW. TBH I am pretty sick of it.
I would like to just talk to Max and have it collaboratively create an environment I want. Can I give an example:
Hi Max, I'd like to mess with some 808 style drums today, can you make me some 808 style trap beats.
Hi, Max ya that's good but I would like the kick to be a little louder and a little more distorted.
Hi Max, can you add in a random 1/8th note bass pattern in the key on Cminor, keep everything in time.
Hi Max, can create populate a latent space full of snares and make a 2d XY pad that lets me move around the snare space.
Hi Max, can you send a message to MYFRIEND and ask them to join in this session.
I would like audio classification, audio translation, transcription, transforming. I want them too to be controlled with whisper or equivalent. I want a completely new modular media environment.
I want to be able to interact with Max in a collaborative way using voice and gesture. I would like it to be zero code as much as possible.
These are my thoughts recently - the higher the technology, the more invisible it should be.
Thanks for everyone's feedback! It's a weighty topic for sure, and one that brings about passionate arguments on all sides.
For me - I'm very fond of the term "Co-Pilot" that Microsoft has been using with many of its new tools, and I for one see the AI tools coming in the near future/here already as helping me in exactly that way. Not fully piloting the plane, but assisting me to fly it (or insert your other appropriate analogy).
Inevitably, I believe the pilot + AI will perform better than the pilot alone....
google bard is now available in germany and i met him last night.
when i asked him for a max patch, he gave me this example:
[in~]
[osc~ 1000]
[out~]
which is undoubtedly pd adn not max.
when i told him it was pd, he apologized.
then i asked him for a hilbert transform and he pointed me to hilbert~. not sure if he means the pd abstraction or the max5 object, but at least it is not totally wrong.
then i tried to explain him that i wanted him to code the hilbert transform himself and not use an external, and he apologized.
to get the answer i wanted, i explicitely asked for a hilbert transform on the base of a polyphase filter.
which seems to be easier that some people think:
[in~]
[poly~ 40]
[hilbert~]
[out~]
i asked him to explain that suggestion and he explained it to me in a very detail:
"the polyphase objects supplies a polyphase filter with 20 phases."
haha. now, this is not even right in pd, not even close. and there is no phaseshift other than 90 degress with this type of filter at all.
speaking of that, he apologized and now suggest me an alternative patch:
[in~ 1]
[fft~ 1024]
[ifft~ 1024]
[abs~]
[tanh~]
[out~ 1]
oha. thanks for stuff i didnt ask for, and for a patch which cant work, but at least he knows that you can - no, you have to - use complex fft to rotate the phase for arbitrary values.
yada yada, then he apologized for his dumbness and then explained me his fft approach:
"the fft~-object performs an fft of the signal, which calculates the hilbert transformation faster as the poly~ object would do."
hmmmm... no, my friend. FFT is the slowest of all and your poly~ doesnt do sh*te.
yada yada, apologizes for him beeing a beta version, and promises future improvements.
after 5 minutes we were at the stage where he insisted that you can patch a 33 step sequencer like so:
[seq 33]
[metro]
and after every excuse for not knowing the answer he brought up the same solution, followed by apologizing.
i asked him not to apologize all the time, and he apologized for apologizing.
you´re better off discussing these matters with your cat or a slice of bread.
LOL, love it Roman! But these are early days still, and despite the obvious errors you point out, I still find all of the above exciting. Think about it - at least it more or less understood what you were asking for and generated output that demonstrated some basic underlying principles of Max programming. More than your cat or bread could do. :-)
compared to max, he is close to perfect in html and css.
the sources (sic!) for html are probably 100 million times more frequently to find, just as for understadning deutsch or english, too, plus the rules of these languages, how you can use the different tags, are very clear and so he almost "understands" this language like we do.
i could not say that his html is any worse that ChatGPT 3.0´s or mine (where i also need a lot of looking into online helps for many things.)
in regards of wikipedia and wolfram alpha he freely admits that he can do what they can do only because they are his main sources for these kind of things. but he didnt want to discuss why i should use him instead of going there directly. :)
ChatGPT, btw., mostly shows me my own patches when i ask him about max. i wonder where he has those from?
can you create me a max/msp patch in max 4 format?
#include "msp.h"
// This patch creates a sine wave and plays it back through the speakers.
// Create a sine wave object.
sine = SinOsc(440);
// Create an audio output object.
Out audioOut = Out(0);
// Route the sine wave to the audio output object.
sine.connect(audioOut);
// Start the patch.
start(44100);
to use this patch, save it as .maxpat-file and then open it in max.
MaxC++ollider v. 4.x/5.x ? :(
So any news on this ? Theres so many possibilities being explored in visual software like touch designer with AI etc (and no the AI doesnt make the artwork for you if you want it to be original- it requires all sorts of human tweakery) - is there much happening in max, or in the pipeline ?? Any new externals using tensorflow or similar ?
We need a lot of patchers and their meta description in human language to teach the text2patcher maschine.
even worse, you would need correctly working patchers.
Thats one use - I personally find patching soothing even though its tedious and solving problems in max is rewarding when you finally get it working. But there's so many other creative possibilities apart from getting it to write patches for you (which would be handy sometimes for sure)
I'm going to take a crack at replying to Wil's comment specifically.
I also welcome debate on this topic; it's certainly a fascinating and complex issue.
In Homo Deus, Yuval Noah Harari discusses how human ears have been found to prefer AI-generated music in some instances. There was even a study where trained musicians either couldn't tell the difference between a computer-generated instrument and a real one or believed the computer-generated instrument was the real thing. This raises intriguing questions about the boundaries between human-made and machine-made art, perhaps suggesting that the rise of AI may not be as much of an issue for artists as it seems.
After reviewing your "art" on YouTube, it appears that the string instrument you're using isn't purely analog as you claim but seems to be digitally processed. Those who actually do work with AI often have to remind people of this point endlessly: the idea that what we perceive as uniquely human—like the ability to forget—may not be as unique as we think. AI can replicate that too, in its own way, by selectively "forgetting" or modifying data to create something new and unpredictable, much like how the ability to forget or make mistakes can lead to unexpected advantages.
While there's certainly a saturation of music today, this shift has also democratized the art form. Is AI really killing art when there are more musicians than ever before using it and creating today? Of course not! From both a market and critical perspective, we should welcome this development for reasons that align with an artistic vision—expanding the boundaries of creativity and inviting more diverse expressions of art. Historically, the transition from analog to MIDI instruments didn't replace analog instruments; it added to the toolkit. Similarly, AI is just another tool that broadens the possibilities. Yelling at fast trains always looks bad in retrospect.
Looking forward to hearing everyone's thoughts.
Cheers,
Karl
I think AI can replace artisans but not artists. The line between those is blurry, but the majority of mainstream music I would locate in the field of artisans.
What the majority of people like or prefer has no correlation with artistic quality. That only time will tell…
And real people need human Heros. That is hard to replace, as AI doesn‘t marry and then jump into some drama to sing about it and give interviews for the tabloid press…
I also think, that AI will be a tool for artists, like any other tool and it is already…
The Italian composer Ivan Fedele once told me "the artisan makes what he knows, the artist makes what he doesn't know".
If we consider that AI systems are trained with the existing repertoire, I believe that they can produce copies (even indistinguishable from human ones) of what has already been done. I don't know if someday they will be able to produce something never heard before, which in art is called invention.
However, I agree that AI systems are excellent tools (but according to Mac Luhan the medium is not neutral, but rather a factor that influences the perception of those who receive it and that can have a profound impact on the culture and society in which it is used).
yup, as riccardo says, it is copies.
copies of everything known, with a strong tendency to the average of everything known.
markov chains on steroids.
it can be indistinguishable from an artwork, so it has the potential to replace products, but it can´t replace artworks.
this A I G PT thing will be the worst thing that ever happens to art.Haha... reply to myself
//
not because it is just a tool to add to the belt, but because babies will grow up to rely more on this tool, and less and less knowing about the shovel
Ask ChatGPT ;-) :
Can AI replace artists ?
The question of whether AI can replace artists is complex and multifaceted, involving not just technological capabilities but also deep philosophical, cultural, and economic considerations. While AI has made significant strides in generating art, music, literature, and other creative outputs, whether it can fully replace artists is still a matter of debate. Below, I will explore the different dimensions of this question.
### 1. **Technological Capabilities of AI in Art**
AI has demonstrated impressive abilities in generating art that mimics human creativity. Technologies like GPT (which powers me), DALL-E, and others have been used to create paintings, music, poetry, and even films. These systems analyze vast datasets of existing artworks to generate new pieces, sometimes even in specific styles or mimicking particular artists. For example, AI has created paintings in the style of Van Gogh or composed music reminiscent of Bach.
These AI systems can produce artworks that are technically proficient and sometimes indistinguishable from those made by humans. This has led to AI-created pieces being sold in galleries and even winning awards in competitions where they were mistaken for human creations.
### 2. **Creativity and Originality**
However, creativity is more than just the ability to produce something novel. It involves the capacity to imbue work with personal experience, emotions, and a sense of purpose. Human artists draw on a lifetime of experiences, emotions, cultural context, and a deep understanding of human nature. They often push boundaries, challenge norms, and evoke emotional responses in ways that are deeply personal and reflective of their inner lives.
AI, on the other hand, lacks consciousness, emotions, and subjective experience. It generates outputs based on patterns in data, not on a genuine understanding of the world or an internal creative impulse. While AI can mimic creativity, it does so by reconfiguring existing elements rather than through a truly original process. This leads to a critical question: Can something be considered "art" if it lacks the intentionality and emotional depth that human artists bring?
### 3. **Cultural and Societal Roles of Art**
Art serves a broad range of functions in society, from expressing individual identity to reflecting collective cultural values, challenging the status quo, and fostering social change. Artists often play a crucial role in interpreting the world around them, offering insights and perspectives that can lead to personal and societal transformation.
AI-generated art can certainly be innovative, but it does not carry the same cultural or societal weight. An AI does not understand the socio-political context in which it operates, nor does it have a personal stake in the outcome of its work. The significance of art often lies in its ability to connect with audiences on a deep, emotional level, something that AI, as it currently stands, is unable to genuinely achieve.
### 4. **Economic Impacts and the Future of Work**
The rise of AI in creative fields does pose significant economic questions. AI can produce art, music, writing, and design at a fraction of the cost and time required by human artists, potentially disrupting traditional creative industries. This could lead to job displacement for some artists, especially in commercial fields like graphic design, content creation, and illustration.
However, this doesn't necessarily mean that AI will fully replace human artists. Instead, it could lead to a redefinition of the role of artists in society. Artists might increasingly collaborate with AI, using these tools to enhance their work, explore new creative possibilities, or automate routine tasks, allowing them to focus on more complex or emotionally resonant aspects of their craft.
### 5. **The Human Element**
At the core of the debate is the irreplaceable human element in art. Art is a form of communication between the artist and the audience, often conveying complex emotions, social commentary, and personal experiences. While AI can simulate certain aspects of creativity, it lacks the consciousness to understand or convey these deeper meanings.
Art is also about the process, the struggle, and the journey of creation. Many artists view their work as an extension of themselves, a means of exploring and understanding their own identities and the world around them. This human-centric aspect of art is something that AI cannot replicate because it does not have an identity, emotions, or a personal connection to the work it creates.
### Conclusion
AI has undoubtedly transformed the creative landscape and will continue to play an increasingly prominent role in art and other creative fields. It can generate art, assist artists, and even push the boundaries of what we consider possible in creative expression. However, replacing human artists entirely is unlikely, at least in the foreseeable future.
Art is inherently tied to human experience, emotions, and consciousness—elements that AI, in its current form, cannot replicate. Instead of replacing artists, AI is more likely to serve as a powerful tool that artists can use to augment their creativity, explore new ideas, and streamline aspects of the creative process. The future of art may involve a symbiosis between human creativity and AI, where both contribute in their unique ways to the evolution of artistic expression.thanks SÉBASTIEN GAY
(that AI machine could have just said 'shovel' and saved a lot of word salad :)
To be honest, I am impressed by its ability to mimic what we could say. Because it is powerful, of course. At the same time, I can't help but think how poorly orignal our thoughts are ... Everything has always already being said on a given subject. The mirror effect is often chilling.