Learning Concurrency in Python



Python is a very high level, general purpose language that is utilized heavily in fields such as data science and research, as well as being one of the top choices for general purpose programming for programmers around the world. It features a wide number of powerful, high and low-level libraries and frameworks that complement its delightful syntax and enable Python programmers to create.

Рейтинг:
Добавить в избранные:
Автор:
Категория: Программирование
Страниц: 562
Скачать Epub файл

1. Preface
2. What this book covers
3. What you need for this book
4. Who this book is for
5. Conventions
6. Reader feedback
7. Downloading the example code
8. Errata
9. Piracy
10. Questions
11. Speed It Up!
12. History of concurrency
13. Threads and multithreading
14. What is a thread?
15. Types of threads
16. What is multithreading?
17. Processes
18. Properties of processes
19. Multiprocessing
20. Event-driven programming
21. Turtle
22. Breaking it down
23. Reactive programming
24. ReactiveX - RxPy
25. Breaking it down
26. GPU programming
27. PyCUDA
28. OpenCL
29. Theano
30. The limitations of Python
31. Jython
32. IronPython
33. Why should we use Python?
34. Concurrent image download
35. Sequential download
36. Breaking it down
37. Concurrent download
38. Breaking it down
39. Improving number crunching with multiprocessing
40. Sequential prime factorization
41. Breaking it down
42. Concurrent prime factorization
43. Breaking it down
44. Summary
45. Parallelize It
46. Understanding concurrency
47. Properties of concurrent systems
48. I/O bottlenecks
49. Understanding parallelism
50. CPU-bound bottlenecks
51. How do they work on a CPU?
52. Single-core CPUs
53. Clock rate
54. Martelli model of scalability
55. Time-sharing - the task scheduler
56. Multi-core processors
57. System architecture styles
58. SISD
59. SIMD
60. MISD
61. MIMD
62. Computer memory architecture styles
63. UMA
64. NUMA
65. Summary
66. Life of a Thread
67. Threads in Python
68. Thread state
69. State flow chart
70. Python example of thread state
71. Breaking it down
72. Different types of threads
73. POSIX threads
74. Windows threads
75. The ways to start a thread
76. Starting a thread
77. Inheriting from the thread class
78. Breaking it down
79. Forking
80. Example
81. Breaking it down
82. Daemonizing a thread
83. Example
84. Breaking it down
85. Handling threads in Python
86. Starting loads of threads
87. Example
88. Breaking it down
89. Slowing down programs using threads
90. Example
91. Breaking it down
92. Getting the total number of active threads
93. Example
94. Breaking it down
95. Getting the current thread
96. Example
97. Breaking it down
98. Main thread
99. Example
100. Breaking it down
101. Enumerating all threads
102. Example
103. Breaking it down
104. Identifying threads
105. Example
106. Breakdown
107. Ending a thread
108. Best practice in stopping threads
109. Example
110. Output
111. Orphan processes
112. How does the operating system handle threads
113. Creating processes versus threads
114. Example
115. Breaking it down
116. Multithreading models
117. One-to-one thread mapping
118. Many-to-one
119. Many-to-many
120. Summary
121. Synchronization between Threads
122. Synchronization between threads
123. The Dining Philosophers
124. Example
125. Output
126. Race conditions
127. Process execution sequence
128. The solution
129. Critical sections
130. Filesystem
131. Life-critical systems
132. Shared resources and data races
133. The join method
134. Breaking it down
135. Putting it together
136. Locks
137. Example
138. Breaking it down
139. RLocks
140. Example
141. Breaking it down
142. Output
143. RLocks versus regular locks
144. Condition
145. Definition
146. Example
147. Our publisher
148. Our subscriber
149. Kicking it off
150. The results
151. Semaphores
152. Class definition
153. Example
154. The TicketSeller class
155. Output
156. Thread race
157. Bounded semaphores
158. Events
159. Example
160. Breaking it down
161. Barriers
162. Example
163. Breaking it down
164. Output
165. Summary
166. Communication between Threads
167. Standard data structures
168. Sets
169. Extending the class
170. Exercise - extending other primitives
171. Decorator
172. Class decorator
173. Lists
174. Queues
175. FIFO queues
176. Example
177. Breaking it down
178. Output
179. LIFO queues
180. Example
181. Breaking it down
182. Output
183. PriorityQueue
184. Example
185. Breakdown
186. Output
187. Queue objects
188. Full/empty queues
189. Example
190. Output
191. The join() function
192. Example
193. Breakdown
194. Output
195. Deque objects
196. Example
197. Breakdown
198. Output
199. Appending elements
200. Example
201. Breaking it down
202. Output
203. Popping elements
204. Example
205. Breaking it down
206. Output
207. Inserting elements
208. Example
209. Breaking it down
210. Output
211. Rotation
212. Example
213. Breaking it down
214. Output
215. Defining your own thread-safe communication structures
216. A web Crawler example
217. Requirements
218. Design
219. Our Crawler class
220. Our starting point
221. Extending the queue object
222. Breaking it down
223. Output
224. Future enhancements
225. Conclusion
226. Exercise - testing your skills
227. Summary
228. Debug and Benchmark
229. Testing strategies
230. Why do we test?
231. Testing concurrent software systems
232. What should we test?
233. Unit tests
234. PyUnit
235. Example
236. Output
237. Expanding our test suite
238. Unit testing concurrent code
239. Integration tests
240. Debugging
241. Make it work as a single thread
242. Pdb
243. An interactive example
244. Catching exceptions in child threads
245. Benchmarking
246. The timeit module
247. Timeit versus time
248. Command-line example
249. Importing timeit into your code
250. Utilizing decorators
251. Timing context manager
252. Output
253. Profiling
254. cProfile
255. Simple profile example
256. The line_profiler tool
257. Kernprof
258. Memory profiling
259. Memory profile graphs
260. Summary
261. Executors and Pools
262. Concurrent futures
263. Executor objects
264. Creating a ThreadPoolExecutor
265. Example
266. Output
267. Context manager
268. Example
269. Output
270. Maps
271. Example
272. Output
273. Shutdown of executor objects
274. Example
275. Output
276. Future objects
277. Methods in future objects
278. The result() method
279. The add_done_callback() method
280. The .running() method
281. The cancel() method
282. The .exception() method
283. The .done() method
284. Unit testing future objects
285. The set_running_or_notify_cancel() method
286. The set_result() method
287. The set_exception() method
288. Cancelling callable
289. Example
290. Output
291. Getting the result
292. Example
293. Output
294. Using as_completed
295. Example
296. Output
297. Setting callbacks
298. Example
299. Output
300. Chaining callbacks
301. Exception classes
302. Example
303. Output
304. ProcessPoolExecutor
305. Creating a ProcessPoolExecutor
306. Example
307. Output
308. Context Manager
309. Example
310. Output
311. Exercise
312. Getting started
313. Improving the speed of computationally bound problems
314. Full code sample
315. Output
316. Improving our crawler
317. The plan
318. New improvements
319. Refactoring our code
320. Storing the results in a CSV file
321. Exercise - capture more info from each page crawl
322. concurrent.futures in Python 2.7
323. Summary
324. Multiprocessing
325. Working around the GIL
326. Utilizing sub-processes
327. Example
328. Output
329. The life of a process
330. Starting a process using fork
331. Spawning a process
332. Forkserver
333. Daemon processes
334. Example
335. Breaking it down
336. Output
337. Identifying processes using PIDs
338. Example
339. Output
340. Terminating a process
341. Example
342. Getting the current process
343. Subclassing processes
344. Example
345. Output
346. Multiprocessing pools
347. The difference between concurrent.futures.ProcessPoolExecutor and Pool
348. Context manager
349. Example
350. Output
351. Submitting tasks to a process pool
352. Apply
353. Apply_async
354. Map
355. Map_async
356. Imap
357. Imap_unordered
358. Starmap
359. Starmap_async
360. Maxtasksperchild
361. Communication between processes
362. Pipes
363. Anonymous pipes
364. Named pipes
365. Working with pipes
366. Example
367. Handling Exceptions
368. Using pipes
369. Multiprocessing managers
370. Namespaces
371. Example
372. Queues
373. Example
374. Output
375. Listeners and clients
376. Example
377. The Listener class
378. The Client class
379. Output
380. Logging
381. Example
382. Communicating sequential processes
383. PyCSP
384. Processes in PyCSP
385. Output
386. Summary
387. Event-Driven Programming
388. Event-driven programming
389. The event loop
390. Asyncio
391. Getting started
392. Event loops
393. The run_forever() method
394. The run_until_complete() method
395. The stop() method
396. The is_closed() method
397. The close() function
398. Tasks
399. Example
400. The all_tasks(loop=None) method
401. The current_tasks() function
402. The cancel() function
403. Task functions
404. The as_completed(fs, *, loop=None, timeout=None) function
405. The ensure_future(coro_or_future, *, loop=None) function
406. The wrap_future(future, *, loop=None) function
407. The gather(*coroes_or_futures, loop=None, return_exceptions=False) function
408. The wait() function
409. Futures
410. Example
411. Output
412. Coroutines
413. Chaining coroutines
414. Output
415. Transports
416. Protocols
417. Synchronization between coroutines
418. Locks
419. Queues
420. Events and conditions
421. Semaphores and BoundedSemaphores
422. Sub-processes
423. Debugging asyncio programs
424. Debug mode
425. Twisted
426. A simple web server example
427. Gevent
428. Event loops
429. Greenlets
430. Simple example-hostnames
431. Output
432. Monkey patching
433. Summary
434. Reactive Programming
435. Basic reactive programming
436. Maintaining purity
437. ReactiveX, or RX
438. Installing RxPY
439. Observables
440. Creating observers
441. Example
442. Example 2
443. Breaking it down
444. Output
445. Lambda functions
446. Example
447. Breaking it down
448. On_next, on_completed, and on_error in lambda form
449. Output
450. Operators and chaining
451. Filter example
452. Breaking it down
453. Chained operators
454. The different operators
455. Creating observables
456. Transforming observables
457. Filtering observables
458. Error-handling observables
459. Hot and cold observables
460. Emitting events
461. Example
462. Breaking it down
463. Output
464. Multicasting
465. Example
466. Output
467. Combining observables
468. Zip() example
469. Output
470. The merge_all() operator
471. Output
472. Concurrency
473. Example
474. Output
475. PyFunctional
476. Installation and official docs
477. Simple example
478. Output
479. Streams, transformations, and actions
480. Filtering lists
481. Output
482. Reading/writing SQLite3
483. Compressed files
484. Parallel execution
485. Summary
486. Using the GPU
487. Basic reactive programming
488. Maintaining purity
489. ReactiveX, or RX
490. Installing RxPY
491. Observables
492. Creating observers
493. Example
494. Example 2
495. Breaking it down
496. Output
497. Lambda functions
498. Example
499. Breaking it down
500. On_next, on_completed, and on_error in lambda form
501. Output
502. Operators and chaining
503. Filter example
504. Breaking it down
505. Chained operators
506. The different operators
507. Creating observables
508. Transforming observables
509. Filtering observables
510. Error-handling observables
511. Hot and cold observables
512. Emitting events
513. Example
514. Breaking it down
515. Output
516. Multicasting
517. Example
518. Output
519. Combining observables
520. Zip() example
521. Output
522. The merge_all() operator
523. Output
524. Concurrency
525. Example
526. Output
527. PyFunctional
528. Installation and official docs
529. Simple example
530. Output
531. Streams, transformations, and actions
532. Filtering lists
533. Output
534. Reading/writing SQLite3
535. Compressed files
536. Parallel execution
537. Summary
538. Choosing a Solution
539. Libraries not covered in this book
540. GPU
541. PyGPU
542. Event-driven and reactive libraries
543. Tornado
544. Flask
545. Celery
546. Data science
547. Pandas
548. Matplotlib
549. TensorFlow
550. Designing your systems
551. Requirements
552. Functional requirements
553. Non-functional requirements
554. Design
555. Computationally expensive
556. Event-heavy applications
557. I/O-heavy applications
558. Recommended design books
559. Software Architecture with Python
560. Python: Master the Art of Design Patterns
561. Research
562. Summary