The concurrency problem of map is not so easy to find, and can be checked by -race. Source code will check whether there is concurrent write, delete the key, traversal when concurrent read and write problems. If this flag exists, a concurrency error will occur.Īfter setting, hashWriting will be reset. The reason is to check the hashWriting flag when reading. This is related to the source code of go. Finally, let's run the code and see the resultsįatal error: concurrent map read and map writeĪs a result, we expect that golang's build in map does not support concurrent read-write operations. The first goroutine is responsible for constantly reading the map object m, while the second goroutine is constantly writing the same data to the map object M. The meaning of the above code is also very easy to understand. Now, based on this scenario, build a sample code The official faq of golang has mentioned that the build in map is not thread(goroutine) safe. ![]() Therefore, the solution before go 1.9 is to bind an additional lock, encapsulate it into a new struct, or use the lock alone. Since go 1.6, concurrent reading and writing of maps will report errors, which exists in some well-known open source libraries. There was no problem with concurrent reading, and there might be problems with concurrent writing. By that time you have both fragments that belong to the pair.Ī typical SAM file for human DNA samples can be around 50 GB compressed and 500 GB uncompressed data, so ensuring that values that are not used anymore can be gc'ed is quite important.Before Go 1.6, the built-in map type was partially goroutine safe. It is known that a particular QNAME only occurs twice in a SAM file, so one way to identify a pair of fragments is by using the QNAME as key in a map and store it the first time you encounter it, and retrieve and delete it again the second time you encounter it. They can be identified by two fields, most importantly the QNAME field (see section 1.4 in the SAM file format). Each "read" consists of two fragments, one at the left end and one at the right end of the read, which are initially stored in FASTQ files, and eventually in SAM files.įor some processing steps in DNA sequencing pipelines, it is important to find the two fragments that belong to a single read. Here are more details about the use case: DNA sequencers use chemical processes to create smaller DNA sequences, which can then be read and stored in files. Maybe it's easier to rename sync.Map to avoid the false impression? However, my understanding was that sync.Map is supposed to be a concurrent version of the Go's built-in map type, and then I believe it should add the functionality I describe above to be complete. Sidenote: I can live with the functionality not being added to sync.Map, because it's not too hard to define a custom Map that implements what is needed. This is also why a solution based on some compare-and-swap functionality is not ideal, because compare-and-swap cannot observe concurrent deletes, it can only observe swaps to some conventional "value-missing" value which doesn't allow the value to be garbage collected. (If there are other references to the value being held by the code that invoked DeleteOrStore, that's fine, it should just be in control of that code.) I care that the storage can be reclaimed by the garbage collector, so the map shouldn't keep a reference to the value that prevents this from happening. I'm not sure I understand this remark correctly. To me, Delete implies that you don't care about the thing being deleted. If the secondary value is not nil, the primary return value is probably meaningless. That's not so different from having a function with a primary return value, and a secondary error value. What happens if you return a non-nil newValue with store = false?) I expected to see at least methods Modify and DeleteOrStore on sync.Map What did you see instead? I don't know good use cases for LoadOrCompute and DeleteOrCompute, though, unlike for Modify and DeleteOrStore. Likewise, a DeleteOrCompute is potentially useful. This could be useful if computing a new value is expensive for some reason.
0 Comments
Leave a Reply. |