How to delete jobs in queue

when an agent was offline and is then online again, then the queued build will be executed. How can we prevent this?

Not out of the box. You want to have this when you take the agent online after you took it offline temporarily? When the agent was disconnected (e.g. because the machine was rebooted) and comes back online and connects again, then no there is now way to prevent this.

No automatic prevention, as Markus said; but if the builds landing on that agent can somehow be distinguished from the others, and you want to do it in some programmatic fashion – you can utilize script console magic. Something like this maybe:

Jenkins.instance.queue.items.each {
  def t = it.task
  println("-- ${it.getClass()} / ${t.getClass()} / ${t} / ${it.assignedLabel}")
  if ("${it.assignedLabel}".toLowerCase().contains('gpu')) {
    println('   cancel')
    Jenkins.instance.queue.cancel(t)
  }
}
-- class hudson.model.Queue$BuildableItem / class org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask / ExecutorStepExecution.PlaceholderTask{label=does-not-exist,context=CpsStepContext[3:node]:sandbox/tmp-random#101} / does-not-exist
-- class hudson.model.Queue$BuildableItem / class org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask / ExecutorStepExecution.PlaceholderTask{label=docker-gpu,context=CpsStepContext[3:node]:sandbox/tmp-random#104} / docker-gpu
   cancel
-- class hudson.model.Queue$BuildableItem / class org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask / ExecutorStepExecution.PlaceholderTask{label="Ubuntu 22 (GPU) - kai",context=CpsStepContext[3:node]:sandbox/tmp-random#103} / Ubuntu 22 (GPU) - kai
   cancel

You can explore BuildableItem and PlaceholderTask and see if any of the fields available (including fields of the fields of their fields) would make sense for you task.