(Generative) AI in Clojure riding on Python Libraries

Table of Contents

• Install relevant python libraries
• Add the library to deps.edn file
• Create a namespace, import Clojure libraries and start coding
• Provide prompt and generate image
• We have the result, so its time to open the surprise box:

Half a decade ago I spent my college years writing Clojure in the context of Data Science and Machine Learning, among the other things. I enjoyed, can’t deny it, but eventually the major library Incanter was deprecated and abandoned. I was left with no easy choice but to jump on a Python bandwagon to keep up with the industry. A couple winters passed, new people took the matter into their own hands, companies funded development of libraries and currently we are experiencing the beginning of new era of data science and machine learning in Clojure.

The power of the language lies in its design that gently guides you to writing concise, performant and reasonable code, and being purely functional, the end result of data exploration is a deployable pipeline instead of mumbled Jupiter notebook. Although ecosystem is getting rich in terms of data science, AI and Machine Learning libraries, Clojure’s symbiotic nature is undeniably handy and brings all the goods of Python at our disposal.

Libpython-clj lets us use any Python library through Clojure, authored by the prolific Clojurian – Chris Nuernberger

But one would ask, including I – why should I use Python library in Clojure if I can simply use Python? What’s the catch?

The answer lies in the combination of functional programming principles, the simplicity inherited from Lisp, and the power of the JVM. This powerful trio called Clojure, allows us to develop software products with a remarkable development experience, minimizing complexity and frustration along the way.

A short detour:

A couple of years ago, one evening my nephew remarked that my face looked happier than usual and he asked – have you been writing Clojure today? And he was correct. I was working on an ML product, building with Python, and sometimes when things got messier in my head, Clojure was the thinking space to shake it off and put things in order. My strategy was to take the problem, write up in Clojure, and eventually re-implement it back in Python. In my personal opinion, Clojure is a thinking tool in the first place.

How does this story relate to our one way conversation?

The only reason I used Python for the project was lack of availability of libraries in Clojure or interoperability. Which no longer holds to be true. Let’s dive into code and see, how can we tame the snake in the domain of parenthesis.

Since generative AI is the hot topic of our recent times, let’s turn on our heterogeneous imagination and hit the REPL.

Install relevant python libraries

pip3 install transformers diffusers ftfy accelerate

Add the library to deps.edn file

{:deps {org.clojure/clojure {:mvn/version "1.11.1"} 
        clj-python/libpython-clj {:mvn/version "2.024"}}}

Create a namespace, import Clojure libraries and start coding

(ns data-science.core
  (:require
    [libpython-clj2.require :refer [require-python]]
    [libpython-clj2.python :as py :refer [py. py.-]]))
	
  (py/from-import diffusers DiffusionPipeline)
	
  (def model "runwayml/stable-diffusion-v1-5")
	
  (def pipeline (-> (py. DiffusionPipeline from_pretrained model)
	            (py. to "mps")))

• :require let’s us import libraries and other namespaces in Clojure
• require-python takes care of python bridge initialization
• py/* are the supporting functions, for example
  • py. takes a method, one or more arguments and executes in Python: (py. obj method arg1 arg2 ... argN)
  • py.- is used for retrieving attributes: (py. obj attr)
• Using our intermediary lib, we can import and employ Python classes(py/from-import diffusers DiffusionPipeline)
  • from-import translates into from x import y
  • import-as is also available and it’s the way of saying import x as y
• (def x) defines a var, in our case model name to retrieve from HuggingFace model repository
• Next, pipeline results in a Python object, but let’s set this 2 lines apart with little more explanation:
  • As we mentioned above (py. ...) let’s us call Python, hence we’re creating DiffusionPipelineobject, invoking a method from_pretrained with the argument string under model.
  • (-> ...) this is called thread first macro, the result of the previous expression is passed to the following one as the first argument:(-> "Hello" (str "from Clojure")) Results in Hello from Clojure, Hello is the first argument of str function.
  • In our case, we invoke another function to in the second step (py. to "mps") which sets hardware type to be used.
    • mps stands for Apple Sillicon
    • cuda is for Nvidia Graphical card
    • cpu for Intel or AMD cpu
  • One caveat of this approach is requirement to download a model, which weighs up to 3 gigs, and will take some time to complete the retrieval, depends on the speed of your internet connection.

Provide prompt and generate image

At this point model is downloaded, configuration applied and we’re ready to start getting results.

(def prompt "A land of Lisp and Clojure, artstation")
(def result (-> prompt pipeline (py.- images))) 
;; =>
;; 0%|          | 0/50 [00:00<?, ?it/s]
;; ...
;; 100%|##########| 50/50 [00:49<00:00,  1.02it/s]

• prompt is the input for the model: (py/callable? pipeline) ;; => true
  • This is the way to check if the object is callable in Clojure, without (py. )
• Let’s go step by step through the process
  • (pipeline prompt) generates the image, it roughly take a minute on my machine – MacBook Pro M1 Pro
  • (py.- images) receives the generated result as the first argument and we retrieve attribute images from the object. Hence, the end result is Python PILobject. We can decompose and check the content of Python objects using py/as-map function:

(py/as-map results) 
;; [<PIL.Image.Image image mode=RGB size=512x512 at 0x307930550>]

We have the result, so its time to open the surprise box:

(py. (first result) show)

---

At this colorful note, we can conclude our first of many upcoming conversations regarding AI, Clojure and Python interoperability. And special thanks to Carin Meier for guiding my way around Libpython-clj and for the knowledge she has shared with the Clojure and AI community over the years.

Cheers,
Giga