Running a pipeline multiple times

Community
1

Hello,

I would like to loop over a number of NetCDF files. I have tried to follow the ‘running the same model repeatedly’ and the shell scripting instructions but can’t quite get there. I have based my pipeline off the scalable-netcdf-pipeline.txt and have read my hazard in using it’s bookmark e.g. input(‘hazard’, name: ‘hazard’). How can I vary the location field in the bookmark? I have tried adding a param.hazard_layer to my model and using input($hazard_layer, name: ‘hazard’) instead but I am getting a ‘could not parse expression’ error which looks like the filename is being passed in without being handled in it’s bookmark.

Hope that makes sense. Thanks again for your help.
Emma

2

Hi Emma,

I suspect this is because there are quotes missing from around the filename, so RiskScape is interpreting it as an expression, rather than a literal string. You can fix this by telling RiskScape that your parameter is text, by editing your project.ini like so:

param.hazard_layer.properties = text

Add that somewhere to the model in the project.ini and try again. When you tell RiskScape that the parameter is text, it will make sure to quote it for you.

Let us know how you get on,
Nick

3

PS you’ll need RiskScape 1.5.0 or greater for this to work!

4

Hi Nick,

Thanks for your advice - just coming back to this as I had been waiting for a Java update!

This worked so far as the different files are now being read in! However I seem to be losing any attributes I had set in the hazard bookmark - as they are NetCDF files I need to set the geometry attribute. Is there any way to vary the hazard layer but still add the attributes I need? I tried adding these in the model section but couldn’t get things working.

Thanks again for your help!
Emma

5

Hi Emma,

This can be a bit tricky… Ideally I’d get you to use the bookmark templates feature, but I’m not 100% we’ve released it yet. Instead, change

input($hazard_layer, name: ‘hazard’)

to be more like

input(
  bookmark(
    $hazard_layer, 
    {
      "set-attribute.id": 'OBJECT_ID' 
      "set-attribute.geom": 'createPoint(lat, lon)', 
      layer: 'example'
    }
  ),
  name: ‘hazard’
)

What I’ve done here is, instead of defining a bookmark in your project.ini to set up the NetCDF data, I’ve built the bookmark in to the pipeline itself using the expression language. You’ll want to copy my example and make sure all the bookmark parameters (apart from location, which gets swapped out as $hazard_layer) get turned in to the set of options given to the bookmark function. These are in my example as "set-attribute.geom" etc. If you’re struggling to get the syntax right, reply back with what your bookmark looks like and I’ll help you tweak the expression.

Good luck,
Nick

6

Hi Nick,

Thank you! This worked perfectly once I’d added my parameters.

Thanks again,
Emma