Hi, this is an open invitation to design schools, art centres and creative education institutions to create a workshop, event, sessions or courses on Machine Learning tools for creatives. Contact, and start a conversation.
You should play with AI, and get familiar with its role in today's way of making and consuming media, shaping our environment and behaviour.
Machine learning tools are essential and powerful for any creative field; from fashion to cinema, from graphic design to sculpture.
let's build something; either a one day intro session, a week-long workshop or a full course.
To have a practical hands-on experience with MachineLearning tools in the creative practice, or to explore conceptual applications of MachineLearning in the creative field.
key concepts to learn/discuss/play: AI literacy / Dataset Bias / AI ethics
AI changes how we create, redefines the role and the tools of the artist and designers.
Synthetic media with machine learning advances at high speed:
from 128x128 pixels images in 2014, -> to full HD detailed images in 2018
From lab to laptop: The time from a research paper is published until it is available as a production tool is shortening.
from Neural StyleTransfer research paper, -> to photoshop effects.
Automating specialized tasks: Background removal
Sessions
Here is a list of resources and projects that should be helpful and inspiring to any creator, designer or artist getting started playing with Machine Learning tools.
Initially compiled for a workshop given at MASSANA Art and Design Center, now this page serves as a repository of concepts, tools and future courses ideas.
Why does it matter?
A preface exercise: With the tools you know and master (photoshop, illustrator, drawing, photography, creative coding…), how would you achieve the following?
Think about the exact steps that you’d take to do this 4 assignments:
With Machine Learning tools, those 4 tasks can be performed automatically.
Tasks like segmentation, object detection, style transfer or image generation are not expensive anymore, allowing the creatives to explore wider..
Now, try this
Key concepts, tools and lectures
ml4a a collection of free educational resources devoted to machine learning for artists. / by Gene Kogan
AI/Machine Learning moments
What is a cat?
What neural networks see
ConvNet Playground An Interactive Visualization Tool for Exploring Convolutional Neural Networks / intro
Empathy with the machine
Name one thing in this photo
Tools & Models
Equipment needed too run ML tools
1) Have your own machine (preferably a linux) with a decent NVIDIA GPU
2) Rent a remote machine by the hours.
3) Run models at Google Colab
Here a list of some Colab Notebooks to experiment with Machine Learning models.
DeepDream by Alexander Mordvintsev
StyleGAN2 by Mikael Christensen
Neural Style Transfer by Titu1994
Ken Burns Effect by Manuel Romero
BigGan by Tensorflow
First Order Motion by Aliaksandr Siarohin
Explore other repositories of Colab Notebooks here and here.
4) use some of the existing tools on the web that are powered by machine learning
5) Use a all-in-one software, like Runway.
Projects & examples
![Anime Faces [+tutorial]](https://images.squarespace-cdn.com/content/v1/52f366cae4b0b5c2c6471998/1590656510643-LOJO7MGQT9LZZ4WZ5RRD/2019-03-17-stylegan-animeface-figure08-truncationtrick.jpg)
Feeding the bots
Machines learn from datasets, those datasets are key to define the model’s output.
If you want to train a model, you need a dataset. you can either use an existing one, or build your own.
Polular image datasets:
like COCO Common Objects in Context.
FFHQ a high-quality image dataset of human faces.
ImageNET, an image database organized according to the WordNet hierarchy
Here is a list of interesting image datasets.
Wikipedia and public domain repositories like natural history illustrations are a good source of content.
For custom image datasets, the easiest approach is to use a browser’s extension to download images.
There are some tools that will help you download images from Google, Bing or Instagram.
Once you gathered the data, it needs to be curated for the model that you want to use it with.
First remove errors (images that do not belong to the concept you are trying to transfer to the machine),
Remove duplicates (dupeGuru is a good desktop tool),
Crop to the requested size by the model
If your dataset is smaller than what you need, you can try to augment it by re-cropping and rotating the images to have larger dataset.
Ethics & Bias
Awful AI is a curated list to track current scary usages of AI - hoping to raise awareness to its misuses in society.
Discrimination
Racist Chat Bots - Microsoft chatbot called Tay spent a day learning from Twitter and began spouting antisemitic messages.
Sexist Recruiting - AI-based recruiting tools such as HireVue or an Amazon internal software, scans various features such as video or voice data of job applicants and their CVs in order to tell whether they're worth hiring. In the case of Amazon, the algorithm quickly taught itself to prefer male candidates over female ones, penalizing CVs that included the word "women's," such as "women's chess club captain." It also reportedly downgraded graduates of two all-women's colleges. [summary][Post article about HireVue]
Influencing, disinformation, and fakes
Deep Fakes - Deep Fakes is an artificial intelligence-based human image synthesis technique. It is used to combine and superimpose existing images and videos onto source images or videos. Deepfakes may be used to create fake celebrity pornographic videos and revenge porn or scam businesses [CNN Interactive Report][Deep Nudes]
Fake News Bots - Automated accounts are being programmed to spread fake news. In recent times, fake news has been used to manipulate stock markets, make people choose dangerous health-care options, and manipulate elections, including the 2016 US presidential election. [summary][NYT Article]
Attention Engineering - From Facebook notifications to Snapstreaks to YouTube auto plays, they're all competing for one thing: your attention. Companies prey on our psychology for their own profit.
Surveillance
Predicting Mass Protests - The US Pentagon funds and uses technologies such as social media surveillance and satellite imagery to forecast civil disobedience and infer location of protesters via their social networks around the world. There are indications that this technology is increasingly used to target Anti-Trump protests, leftwing groups and activists of color.
Palantir - A billion-dollar startup that focuses on predictive policies, intelligence and ai-powered military defense systems. [summary]
Censorship - WeChat, a messaging app used by millions of people in China, uses automatic analysis to censor text and images within private messaging in real-time. Using optical character recognition, the images are examined for harmful content — including anything about international or domestic politics deemed undesirable by the Chinese Communist Party. It’s a self-reinforcing system that’s growing with every image sent. [research summary]
Social credit systems
Social Credit System - Using a secret algorithm, Sesame credit constantly scores people from 350 to 950, and its ratings are based on factors including considerations of “interpersonal relationships” and consumer habits. [summary][Foreign Correspondent (video)][travel ban]
Health Insurance Credit System - Health insurance companies such as Vitality offer deals based on access to data from fitness trackers. However, they also can charge more and even remove access to important medical devices if patients are determined to be non compliant to unfair pricing. [ProPublica]
Contestational research
Research to create a less awful and more privacy-preserving AI
Differential Privacy - A formal definition of privacy that allows us to make theoretical guarantees on data breaches. AI algorithms can be trained to be differentially private. [original paper]
Fair Machine Learning & Algorithm Bias - A subfield in AI that investigates different fairness criteria and algorithm bias. A recent best paper (in ICLR18), e.g. shows that implementing specific criteria can have a delayed impact on fairness.
Contestational tech projects
These open source projects try to spur discourse, offer protection or awareness to awful AI
AdNauseam - AdNauseam is a lightweight browser extension to fight back against tracking by advertising networks. It works like an ad-blocker (it is built atop uBlock-Origin) to silently simulate clicks on each blocked ad, confusing trackers as to one's real interests. [code]
TrackMeNot - TrackMeNot is a browser extension (Chrome, Firefox) that helps protect your online searches by creating fake search queries. This creates noise in data that makes it harder to track and profile user behaviour. [code]
Visit Awful AI for the extended list.
Datasets & Face Recognition
MegaPixels is an art and research publication investigating the ethics, origins, and individual privacy implications of face recognition datasets created "in the wild.
One of the 11,917 images in the Brainwash dataset captured from the Brainwash Cafe in San Francisco
AI Fails
What happens when AI fails on the systems powering healthcare, finances or predpol?
Asymmetry / Straight hair looks like paint / Text is indecipherable / Background is surreal / Weird teeth / Messy hair / Non-stereotypical gender presentation / Semi-regular noise / Iridescent color bleed.