print("Now the show begins!") pool = multiprocessing.Pool() tasks = 100 results = [] for i inrange(0, tasks): pool.apply_async(task, args=(i,), callback=results.append) pool.close() pool.join() for num, msg in results: print(num, msg)
But the output I got is
1 2
Now the show begins!
The program hangs here and no task is executed at all.
After Google, I realise to make the code work in Windows I must wrap everything in main() and add if __name__ == "__main__ entry, as the multiprocessing doc explained.
defmain(): print("Now the show begins!") pool = multiprocessing.Pool() tasks = 100 results = [] for i inrange(0, tasks): pool.apply_async(task, args=(i,), callback=results.append) pool.close() pool.join() for num, msg in results: print(num, msg)
if __name__ == '__main__': main()
Now everything should work. But still the program hangs after I press the magic ▶ button.
The problem comes from the multiprocessing library, it does not support Pycharm’s venv intepreter. There are two methods to solve the problem.
Run program in cmd instead. (Extremely painful)
Change the interpreter in Run/Debug Configuration from venv to the python installed in the system. (Need install modules)
Uh oh~ logging not working!
While print() always print, logging can be troublesome in multiprocessing. In the following example, the logging works without problem:
defmain(): logger.info("Now the show begins!") pool = multiprocessing.Pool() tasks = 100 results = [] for i inrange(0, tasks): pool.apply_async(task, args=(i,), callback=results.append) pool.close() pool.join() for num, msg in results: print(num, msg)
if __name__ == '__main__': logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) main()
but when attempting to log something inside task for example:
defmain(): logger.info("Now the show begins!") pool = multiprocessing.Pool() tasks = 100 results = [] for i inrange(0, tasks): pool.apply_async(task, args=(i,), callback=results.append) pool.close() pool.join() for num, msg in results: print(num, msg)
if __name__ == '__main__': main()
This works fine.
Logging to the same file in multiprocessing is not safe anymore, but this issue is not discussed here since tons of StackOverflow answer are already good enough.