Towards Reverse Engineering Matplotlib Code From Images : Chris

Towards Reverse Engineering Matplotlib Code From Images
by: Chris
blow post content copied from  Be on the Right Side of Change
click here to view original post

5/5 - (1 vote)

I tried a few helpful applications of Google Bards Image Recognition capabilities for coders. I don’t know about you but I often see beautiful plots (e.g., in research papers or data science reports) and wonder how I could recreate them.

Well, Google Bard to the rescue!

Reverse Engineer Exponential Plot in 2D

First, let’s try a simple exponential curve.

Input Image:

I passed this chart along with the following prompt in Google Bard:

 Prompt: Reverse engineer the code leading to this graphic

So Google Bard suggests the following code to generate the graph in Python.

Bard Output Code:

import matplotlib.pyplot as plt
import numpy as np

def exponential_function(x):
  return np.exp(x)

x_values = np.linspace(0, 10, 100)
y_values = exponential_function(x_values)

plt.plot(x_values, y_values)
plt.xlabel('x')
plt.ylabel('y')
plt.title('Exponential Curve')
plt.show()

This code from Google Bard generates the following chart.

Insane! Google Bard figured out the exact exponential curve by looking at the input chart.

This is really advanced if you think about it. I was particularly impressed by the fact that Bard even got the function right leading to the chart — not only the rough code to plot a similar chart.

So if you needed one more proof of the deep intelligence of large language models (LLMs) that’s way beyond the trivial probabilistic prediction of the next word or character, here it is.

Let’s try another more complex (higher-order) curve:

Reverse Engineer Higher-Order Curve

Now let’s use a higher-order cubic function plot and pass it into Google Bard to see if it can generate the code that produces it.

Input Image:

 Prompt: Give me the code leading to this output

Google Bard spits out the following code.

Bard Output Code:

import matplotlib.pyplot as plt
import numpy as np

# Create the data
x = np.linspace(-10, 10, 100)
y = x**2

# Plot the data
plt.plot(x, y)

# Add a title
plt.title('Plot of a Higher-Order Function')

# Add labels to the axes
plt.xlabel('X')
plt.ylabel('f(x)')

# Show the plot
plt.show()

This yields the following image:

Some things are correct, such as the x-axis and the labels. Other things are incorrect or imprecise such as the y-axis and the plot — even though it figured out that it’s a higher-order function.

3D Normal Distribution

Next, I tried a more complex 3D plot of a multivariate normal distribution. You can read my blog tutorial on how to do this here.

Recommended: How to Plot a 3D Normal Distribution in Python?

This is the image I passed into Google Bard.

Input Image:

And the Bard prompt:

 Prompt: Reproduce the code that generates the image

Bard Output Code:

import plotly.graph_objects as go

fig = go.Figure(data=[go.Cone(
    x=[1, 2, 3],
    y=[1, 2, 3],
    z=[1, 2, 3],
    u=[1, 2, 3],
    v=[1, 1, 2],
    w=[4, 4, 1],
    showscale=False,
    hoverinfo="u+v+w+name",
)])

fig.update_layout(
    title="3D Cone Plot",
    scene=dict(
        camera=dict(
            eye=dict(x=-1.57, y=1.36, z=0.58),
        ),
        domain=dict(x=[0, 1]),
    ),
)

fig.show()

When I run the code, I get the following output:

Fail.

While it captures that the plot is generated with Python and it contains a 3D plot (out of all things possible), it doesn’t produce the correct plot in that case.

Putting It Together

I’m very impressed with Google Bard’s capabilities, and I’m sure they will grow significantly over the next couple of months. Google Bard can already match the capabilities of an entry-level data scientist.

Soon you’ll be able to pass screenshots of arbitrary plots into it and let it generate the code for you. This is six-figure-per-year work currently done by advanced data scientists.

When I was still a researcher at University, I’d have done almost everything for such a tool — given that most research papers live and die by the high-quality plots they generate and most researchers and data scientists spend huge amounts of time figuring out how others have generated their stunning-looking plots.

Recommended: Google Bard Extracts Code From Images (PNG, JPEG, WebP)

July 23, 2023 at 06:33PM
Click here for more details...

=============================
The original post is available in Be on the Right Side of Change by Chris
this post has been published as it is through automation. Automation script brings all the top bloggers post under a single umbrella.
The purpose of this blog, Follow the top Salesforce bloggers and collect all blogs in a single place through automation.
============================