Hfrxdjt

I am sometimes in the following situation:

• I want to execute two cells: Cell 1 takes on the order of 10 minutes or hours. Cell 2 will take 1 second.




• I'd rather execute cell 2 first and see the result, but I didn't know that beforehand and hence already started cell 1.




• cell 1 is already significantly into the computation, so it would be wasteful to just abort it.

How do I first pause cell 1, then start cell 2, and then start cell 1 again? Or even better, how do I pause cell 1, start cell 2, and have cell 1 automatically start again when cell 2 is done?

I don't think Jupyter notebooks (even via extensions) currently offer pausing/restarting cell blocks. I would suggest putting the code of both cells into a single cell and using Python logic to determine the order of execution.

In general, however, you cannot strictly pause the execution of something and come back to it later. If you abort the execution of the function, the intermediate results are lost (because they are stored within the namespace of the function you effectively just killed). This is because Python only runs one single process at a time (key term: Global Interpreter Lock).

The only thing you might be able to do is create a cell 0, which performs some kind of check, testing how long cell 1 might take to run, then just put cell 1 and cell 2 in and if/else construction that gives the correct order to use.

Other approaches

You might want to look into something like the multiprocessing library. There you can create a group of worker processors, to which you can send the contents of cell 1 and at the same time the contents of cell 2. They will be computed at the same time, using two different processes i.e. two different instances of the Python interpreter. This can be easy to achieve if there is no direct dependency between (in your case) cell 1 and cell 2. Multiprocessing is particularly useful in the case that your long-running cell is compute bound, meaning is has to perform a large computation.

Yet another option would be to investigate the threading module and concurrent programming in general, but this get a little more complicated and is probably beyond what you want in your situation (it also requires more effort to get working that multiprocessing). This allows Python to kind of do two things at once, but with shared state - so each running thread can change variables that the other thread might also be changing, which can require a lot of work to make safe. This approach is however beneficial when you tasks are IO bound, i.e. there is no big computation, but rather you send data e.g. to a website and wait for a return - most your time is spent waiting.