For anyone else wanting to do this I figured it out.

First I got the color curve I wanted, and clicked the plus to "Save the current settings as named preset"

Then next to the plus I pressed Manage presets -> export current settings to file

Then I downloaded CurveBatch from here http://gimpfr.org/contrib_photolabo.php

Then in GIMP I clicked edit->Preferences->Folders->Plug-ins to find where plug-ins are saved.

I then saved CurveBatch here after extracting it.

Then for GIMP 2.10 I had to change the CurveBatch plug-in (photolab_curvebatch) slightly

I changed

  if lines[0] == "# GIMP curves tool settings\n":

to

  if lines[0] == "# GIMP 'Curves' settings\n":

As GIMP 2.10 changed the first line that indicates it is a curve file (Why anyone would think for a second to do this and break backwards compatibility for no reason I do not know)

Then I pressed Filters -> Script-fu -> refresh scripts

Then I closed GIMP and reopened it.

Then the script now showed up in filters, I pressed Filters->PhotoLab->Batch corrections->curve

I chose the folder my images are in, their extension and the curve file that I exported previously.

This then applied the curves to all of my pictures as I wanted.