• play around with already made code
• [x] Research and find a project (experiments, websites, art installations, games, etc) that utilizes machine learning in a creative way. Consider the following:
  • What type of machine learning models did the creator use?
  • What data might have been used to train the machine learning model?
  • Why did the creator of the project choose to use this machine learning model?
• [x] Pick one of the models above (FaceMesh or HandPose), following the examples above and the ml5.js documentation, experiment with controlling elements of a p5.js sketch (color, geometry, sound, text, etc) with the output of the model. Try to create an interaction that is surprising or one that is inspired by the project you find.
• [x] Document your research, your response, and your p5.js sketch in a blog post. Add a link to the post and your p5.js sketch on the Assignment 4 Wiki page. In your blog post, include visual documentation such as a recorded screen capture / video / GIFs of your sketch.

Research

Refik Anadol. Unsupervised — Machine Hallucinations — MoMA. 2022.

Refik Anadol. Unsupervised — Machine Hallucinations — MoMA. 2022.

Anadol’s Website & documentation on this project:

https://refikanadol.com/works/unsupervised/

Interesting Article I referenced about this Art piece:

https://amplify.nabshow.com/articles/machine-learning-moma-refik-anadol/

My reflection of this project:

When one of my closest friends starting working for the MoMA, she took me a private tour of the MoMa and I could not forget this one piece. Having transitioned from computer science to Interactive Media Arts (IMA), I found this integration particularly fascinating. The project draws from 380,000 images of 180,000 artworks from MoMA’s collection, which reminded me of ImageNet’s large-scale dataset of over 14 million images and 21,841 synsets. While ImageNet is larger in scale, this project stands out for its unique connection to MoMA’s own art history, showcasing works that were previously exhibited at the museum.

Adrian Pennington’s article on the AI artwork highlights Anadol’s exploration of whether machines can learn, dream, or hallucinate—questions that go beyond the traditional goals of programming. Many programmers focus on creating software for specific products or features, but Anadol’s approach pushes these boundaries. Initially, I questioned whether this project was truly AI or machine learning, but I came to realize that, despite being programmed to produce an output, it also creates an evolving artwork that blends past and present MoMA collections into a program.

What type of machine learning models did the creator use?

According to the article, "The team created and tested various AI models to see which one produced the best results, then picked one and trained it for three weeks." They used image-based machine learning models such as ResNet, known for its deep learning-based structure and functionality. In "ResNeXt: A New Paradigm in Image Processing," Atakan Erdoğan mentions, "ResNeXt is a Convolutional Neural Network (CNN) architecture, which is a deep learning model." ResNeXt can be applied in various areas, particularly deep learning problems involving visual and text data, such as image classification, object detection, face recognition, natural language processing (NLP), and medical image analysis.

What is ResNet? https://medium.com/@atakanerdogan305/resnext-a-new-paradigm-in-image-processing-ee40425aea1f

What data might have been used to train the machine learning model?

The data used to train this machine model consisted of 380,000 images of 138,151 art pieces from MoMA’s collection, creating a stream of moving images. Using ResNet, they developed a machine learning algorithm and implemented custom software for the project.