--- /dev/null
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+<meta name="generator" content="HTML Tidy for Linux/x86 (vers 12 April 2005), see www.w3.org" />
+<title>Associative-Container Performance Tests</title>
+<meta http-equiv="Content-Type" content="text/html; charset=us-ascii" />
+</head>
+<body>
+<div id="page">
+<h1><a name="assoc" id="assoc">Associative-Container
+ Performance Tests</a></h1>
+<h2><a name="settings" id="settings">Settings</a></h2>
+<p>This section describes performance tests and their results.
+ In the following, <a href="#gcc"><u>g++</u></a>, <a href="#msvc"><u>msvc++</u></a>, and <a href="#local"><u>local</u></a> (the build used for generating this
+ documentation) stand for three different builds:</p>
+<div id="gcc_settings_div">
+<div class="c1">
+<h3><a name="gcc" id="gcc"><u>g++</u></a></h3>
+<ul>
+<li>CPU speed - cpu MHz : 2660.644</li>
+<li>Memory - MemTotal: 484412 kB</li>
+<li>Platform -
+ Linux-2.6.12-9-386-i686-with-debian-testing-unstable</li>
+<li>Compiler - g++ (GCC) 4.0.2 20050808 (prerelease)
+ (Ubuntu 4.0.1-4ubuntu9) Copyright (C) 2005 Free Software
+ Foundation, Inc. This is free software; see the source
+ for copying conditions. There is NO warranty; not even
+ for MERCHANTABILITY or FITNESS FOR A PARTICULAR
+ PURPOSE.</li>
+</ul>
+</div>
+<div class="c2"></div>
+</div>
+<div id="msvc_settings_div">
+<div class="c1">
+<h3><a name="msvc" id="msvc"><u>msvc++</u></a></h3>
+<ul>
+<li>CPU speed - cpu MHz : 2660.554</li>
+<li>Memory - MemTotal: 484412 kB</li>
+<li>Platform - Windows XP Pro</li>
+<li>Compiler - Microsoft (R) 32-bit C/C++ Optimizing
+ Compiler Version 13.10.3077 for 80x86 Copyright (C)
+ Microsoft Corporation 1984-2002. All rights
+ reserved.</li>
+</ul>
+</div>
+<div class="c2"></div>
+</div>
+<div id="local_settings_div"><div style = "border-style: dotted; border-width: 1px; border-color: lightgray"><h3><a name = "local"><u>local</u></a></h3><ul>
+<li>CPU speed - cpu MHz : 2250.000</li>
+<li>Memory - MemTotal: 2076248 kB</li>
+<li>Platform - Linux-2.6.16-1.2133_FC5-i686-with-redhat-5-Bordeaux</li>
+<li>Compiler - g++ (GCC) 4.1.1 20060525 (Red Hat 4.1.1-1)
+Copyright (C) 2006 Free Software Foundation, Inc.
+This is free software; see the source for copying conditions. There is NO
+warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+</li>
+</ul>
+</div><div style = "width: 100%; height: 20px"></div></div>
+<h2><a name="assoc_tests" id="assoc_tests">Tests</a></h2>
+<h3><a name="hash_based" id="hash_based">Hash-Based
+ Containers</a></h3>
+<ol>
+<li><a href="hash_text_find_find_timing_test.html">Hash-Based
+ Text <tt>find</tt> Find Timing Test</a></li>
+<li><a href="hash_random_int_find_find_timing_test.html">Hash-Based
+ Random-Integer <tt>find</tt> Find Timing Test</a></li>
+<li><a href="hash_random_int_subscript_find_timing_test.html">Hash-Based
+ Random-Integer Subscript Find Timing Test</a></li>
+<li><a href="hash_random_int_subscript_insert_timing_test.html">Hash-Based
+ Random-Integer Subscript Insert Timing Test</a></li>
+<li><a href="hash_zlob_random_int_find_find_timing_test.html">Hash-Based
+ Skewed-Distribution Random-Integer <tt>find</tt> Find Timing
+ Test</a></li>
+<li><a href="hash_random_int_erase_mem_usage_test.html">Hash-Based Erase
+ Memory Use Test</a></li>
+</ol>
+<h3><a name="tree_like_based" id="tree_like_based">Tree-Like-Based Containers</a></h3>
+<ol>
+<li><a href="tree_text_insert_timing_test.html">Tree-Based
+ and Trie-Based Text Insert Timing Test</a></li>
+<li><a href="tree_text_find_find_timing_test.html">Tree-Based
+ and Trie-Based Text <tt>find</tt> Find Timing Test</a></li>
+<li><a href="tree_text_lor_find_find_timing_test.html">Tree-Based
+ Locality-of-Reference Text <tt>find</tt> Find Timing
+ Test</a></li>
+<li><a href="tree_random_int_find_find_timing_test.html">Tree-Based
+ Random-Integer <tt>find</tt> Find Timing Test</a></li>
+<li><a href="tree_split_join_timing_test.html">Tree Split and
+ Join Timing Test</a></li>
+<li><a href="tree_order_statistics_timing_test.html">Tree
+ Order-Statistics Timing Test</a></li>
+</ol>
+<h3><a name="multimaps" id="multimaps">Multimaps</a></h3>
+<ol>
+<li><a href="multimap_text_find_timing_test_small.html">"Multimap"
+ Text Find Timing Test with <u>Small</u> Average Secondary-Key
+ to Primary-Key Ratio</a></li>
+<li><a href="multimap_text_find_timing_test_large.html">"Multimap"
+ Text Find Timing Test with <u>Large</u> Average Secondary-Key
+ to Primary-Key Ratio</a></li>
+<li><a href="multimap_text_insert_timing_test_small.html">"Multimap"
+ Text Insert Timing Test with <u>Small</u> Average
+ Secondary-Key to Primary-Key Ratio</a></li>
+<li><a href="multimap_text_insert_timing_test_large.html">"Multimap"
+ Text Insert Timing Test with <u>Large</u> Average
+ Secondary-Key to Primary-Key Ratio</a></li>
+<li><a href="multimap_text_insert_mem_usage_test_small.html">"Multimap"
+ Text Insert Memory-Use Test with <u>Small</u> Average
+ Secondary-Key to Primary-Key Ratio</a></li>
+<li><a href="multimap_text_insert_mem_usage_test_large.html">"Multimap"
+ Text Insert Memory-Use Test with <u>Large</u> Average
+ Secondary-Key to Primary-Key Ratio</a></li>
+</ol>
+<h2><a name="assoc_observations" id="assoc_observations">Observations</a></h2>
+<h3><a name="dss_family_choice" id="dss_family_choice">Underlying Data-Structure Families</a></h3>
+<p>In general, hash-based containers (see <a href="hash_based_containers.html">Design::Associative
+ Containers::Hash-Based Containers</a>) have better timing
+ performance than containers based on different underlying-data
+ structures. The main reason to choose a tree-based (see
+ <a href="tree_based_containers.html">Design::Associative
+ Containers::Tree-Based Containers</a>) or trie-based container
+ (see <a href="trie_based_containers.html">Design::Associative
+ Containers::Trie-Based Containers</a>) is if a byproduct of the
+ tree-like structure is required: either order-preservation, or
+ the ability to utilize node invariants (see <a href="tree_based_containers.html#invariants">Design::Associative
+ Containers::Tree-Based Containers::Node Invariants</a> and
+ <a href="trie_based_containers.html#invariants">Design::Associative
+ Containers::Trie-Based Containers::Node Invariants</a>). If
+ memory-use is the major factor, an ordered-vector tree (see
+ <a href="tree_based_containers.html">Design::Associative
+ Containers::Tree-Based Containers</a>) gives optimal results
+ (albeit with high modificiation costs), and a list-based
+ container (see <a href="lu_based_containers.html">Design::Associative
+ Containers::List-Based Containers</a>) gives reasonable
+ results.</p>
+<h3><a name="hash_based_types" id="hash_based_types">Hash-Based
+ Container Types</a></h3>
+<p>Hash-based containers are typically either collision
+ chaining or probing (see <a href="hash_based_containers.html">Design::Associative
+ Containers::Hash-Based Containers</a>). Collision-chaining
+ containers are more flexible internally, and so offer better
+ timing performance. Probing containers, if used for simple
+ value-types, manage memory more efficiently (they perform far
+ fewer allocation-related calls). In general, therefore, a
+ collision-chaining table should be used. A probing container,
+ conversely, might be used efficiently for operations such as
+ eliminating duplicates in a sequence, or counting the number of
+ occurrences within a sequence. Probing containers might be more
+ useful also in multithreaded applications where each thread
+ manipulates a hash-based container: in the STL, allocators have
+ class-wise semantics (see [<a href="references.html#meyers96more">meyers96more</a>] - Item 10); a
+ probing container might incur less contention in this case.</p>
+<h3><a name="hash_based_policies" id="hash_based_policies">Hash-Based Containers' Policies</a></h3>
+<p>In hash-based containers, the range-hashing scheme (see
+ <a href="hash_based_containers.html#hash_policies">Design::Associative
+ Containers::Hash-Based Containers::Hash Policies</a>) seems to
+ affect performance more than other considerations. In most
+ settings, a mask-based scheme works well (or can be made to
+ work well). If the key-distribution can be estimated a-priori,
+ a simple hash function can produce nearly uniform hash-value
+ distribution. In many other cases (<i>e.g.</i>, text hashing,
+ floating-point hashing), the hash function is powerful enough
+ to generate hash values with good uniformity properties
+ [<a href="references.html#knuth98sorting">knuth98sorting</a>];
+ a modulo-based scheme, taking into account all bits of the hash
+ value, appears to overlap the hash function in its effort.</p>
+<p>The range-hashing scheme determines many of the other
+ policies (see <a href="hash_based_containers.html#policy_interaction">Design::Hash-Based
+ Containers::Policy Interaction</a>). A mask-based scheme works
+ well with an exponential-size policy (see <a href="hash_based_containers.html#resize_policies">Design::Associative
+ Containers::Hash-Based Containers::Resize Policies</a>) ; for
+ probing-based containers, it goes well with a linear-probe
+ function (see <a href="hash_based_containers.html#hash_policies">Design::Associative
+ Containers::Hash-Based Containers::Hash Policies</a>).</p>
+<p>An orthogonal consideration is the trigger policy (see
+ <a href="hash_based_containers.html#resize_policies">Design::Associative
+ Containers::Hash-Based Containers::Resize Policies</a>). This
+ presents difficult tradeoffs. <i>E.g.</i>, different load
+ factors in a load-check trigger policy yield a
+ space/amortized-cost tradeoff.</p>
+<h3><a name="tree_like_based_types" id="tree_like_based_types">Tree-Like-Based Container
+ Types</a></h3>
+<p>In general, there are several families of tree-based
+ underlying data structures: balanced node-based trees
+ (<i>e.g.</i>, red-black or AVL trees), high-probability
+ balanced node-based trees (<i>e.g.</i>, random treaps or
+ skip-lists), competitive node-based trees (<i>e.g.</i>, splay
+ trees), vector-based "trees", and tries. (Additionally, there
+ are disk-residing or network-residing trees, such as B-Trees
+ and their numerous variants. An interface for this would have
+ to deal with the execution model and ACID guarantees; this is
+ out of the scope of this library.) Following are some
+ observations on their application to different settings.</p>
+<p>Of the balanced node-based trees, this library includes a
+ red-black tree (see <a href="tree_based_containers.html">Design::Associative
+ Containers::Tree-Based Containers</a>), as does STL (in
+ practice). This type of tree is the "workhorse" of tree-based
+ containers: it offers both reasonable modification and
+ reasonable lookup time. Unfortunately, this data structure
+ stores a huge amount of metadata. Each node must contain,
+ besides a value, three pointers and a boolean. This type might
+ be avoided if space is at a premium [<a href="references.html#austern00noset">austern00noset</a>].</p>
+<p>High-probability balanced node-based trees suffer the
+ drawbacks of deterministic balanced trees. Although they are
+ fascinating data structures, preliminary tests with them showed
+ their performance was worse than red-black trees. The library
+ does not contain any such trees, therefore.</p>
+<p>Competitive node-based trees have two drawbacks. They are
+ usually somewhat unbalanced, and they perform a large number of
+ comparisons. Balanced trees perform one comparison per each
+ node they encounter on a search path; a splay tree performs two
+ comparisons. If the keys are complex objects, <i>e.g.</i>,
+ <tt>std::string</tt>, this can increase the running time.
+ Conversely, such trees do well when there is much locality of
+ reference. It is difficult to determine in which case to prefer
+ such trees over balanced trees. This library includes a splay
+ tree (see <a href="tree_based_containers.html">Design::Associative
+ Containers::Tree-Based Containers</a>).</p>
+<p>Ordered-vector trees (see <a href="tree_based_containers.html">Design::Associative
+ Containers::Tree-Based Containers</a>) use very little space
+ [<a href="references.html#austern00noset">austern00noset</a>].
+ They do not have any other advantages (at least in this
+ implementation).</p>
+<p>Large-fan-out PATRICIA tries (see <a href="trie_based_containers.html">Design::Associative
+ Containers::Trie-Based Containers</a>) have excellent lookup
+ performance, but they do so through maintaining, for each node,
+ a miniature "hash-table". Their space efficiency is low, and
+ their modification performance is bad. These tries might be
+ used for semi-static settings, where order preservation is
+ important. Alternatively, red-black trees cross-referenced with
+ hash tables can be used. [<a href="references.html#okasaki98mereable">okasaki98mereable</a>]
+ discusses small-fan-out PATRICIA tries for integers, but the
+ cited results seem to indicate that the amortized cost of
+ maintaining such trees is higher than that of balanced trees.
+ Moderate-fan-out trees might be useful for sequences where each
+ element has a limited number of choices, <i>e.g.</i>, DNA
+ strings (see <a href="assoc_examples.html#trie_based">Examples::Associative
+ Containers::Trie-Based Containers</a>).</p>
+<h3><a name="msc" id="msc">Mapping-Semantics
+ Considerations</a></h3>
+<p>Different mapping semantics were discussed in <a href="motivation.html#assoc_mapping_semantics">Motivation::Associative
+ Containers::Alternative to Multiple Equivalent Keys</a> and
+ <a href="tutorial.html#assoc_ms">Tutorial::Associative
+ Containers::Associative Containers Others than Maps</a>. We
+ will focus here on the case where a keys can be composed into
+ primary keys and secondary keys. (In the case where some keys
+ are completely identical, it is trivial that one should use an
+ associative container mapping values to size types.) In this
+ case there are (at least) five possibilities:</p>
+<ol>
+<li>Use an associative container that allows equivalent-key
+ values (such as <tt>std::multimap</tt>)</li>
+<li>Use a unique-key value associative container that maps
+ each primary key to some complex associative container of
+ secondary keys, say a tree-based or hash-based container (see
+ <a href="tree_based_containers.html">Design::Associative
+ Containers::Tree-Based Containers</a> and <a href="hash_based_containers.html">Design::Associative
+ Containers::Hash-Based Containers</a>)</li>
+<li>Use a unique-key value associative container that maps
+ each primary key to some simple associative container of
+ secondary keys, say a list-based container (see <a href="lu_based_containers.html">Design::Associative
+ Containers::List-Based Containers</a>)</li>
+<li>Use a unique-key value associative container that maps
+ each primary key to some non-associative container
+ (<i>e.g.</i>, <tt>std::vector</tt>)</li>
+<li>Use a unique-key value associative container that takes
+ into account both primary and secondary keys.</li>
+</ol>
+<p>We do not think there is a simple answer for this (excluding
+ option 1, which we think should be avoided in all cases).</p>
+<p>If the expected ratio of secondary keys to primary keys is
+ small, then 3 and 4 seem reasonable. Both types of secondary
+ containers are relatively lightweight (in terms of memory use
+ and construction time), and so creating an entire container
+ object for each primary key is not too expensive. Option 4
+ might be preferable to option 3 if changing the secondary key
+ of some primary key is frequent - one cannot modify an
+ associative container's key, and the only possibility,
+ therefore, is erasing the secondary key and inserting another
+ one instead; a non-associative container, conversely, can
+ support in-place modification. The actual cost of erasing a
+ secondary key and inserting another one depends also on the
+ allocator used for secondary associative-containers (The tests
+ above used the standard allocator, but in practice one might
+ choose to use, <i>e.g.</i>, [<a href="references.html#boost_pool">boost_pool</a>]). Option 2 is
+ definitely an overkill in this case. Option 1 loses out either
+ immediately (when there is one secondary key per primary key)
+ or almost immediately after that. Option 5 has the same
+ drawbacks as option 2, but it has the additional drawback that
+ finding all values whose primary key is equivalent to some key,
+ might be linear in the total number of values stored (for
+ example, if using a hash-based container).</p>
+<p>If the expected ratio of secondary keys to primary keys is
+ large, then the answer is more complicated. It depends on the
+ distribution of secondary keys to primary keys, the
+ distribution of accesses according to primary keys, and the
+ types of operations most frequent.</p>
+<p>To be more precise, assume there are <i>m</i> primary keys,
+ primary key <i>i</i> is mapped to <i>n<sub>i</sub></i>
+ secondary keys, and each primary key is mapped, on average, to
+ <i>n</i> secondary keys (<i>i.e.</i>,
+ <i><b>E</b>(n<sub>i</sub>) = n</i>).</p>
+<p>Suppose one wants to find a specific pair of primary and
+ secondary keys. Using 1 with a tree based container
+ (<tt>std::multimap</tt>), the expected cost is
+ <i><b>E</b>(Θ(log(m) + n<sub>i</sub>)) = Θ(log(m) +
+ n)</i>; using 1 with a hash-based container
+ (<tt>std::tr1::unordered_multimap</tt>), the expected cost is
+ <i>Θ(n)</i>. Using 2 with a primary hash-based container
+ and secondary hash-based containers, the expected cost is
+ <i>O(1)</i>; using 2 with a primary tree-based container and
+ secondary tree-based containers, the expected cost is (using
+ the Jensen inequality [<a href="references.html#motwani95random">motwani95random</a>])
+ <i><b>E</b>(O(log(m) + log(n<sub>i</sub>)) = O(log(m)) +
+ <b>E</b>(O(log(n<sub>i</sub>)) = O(log(m)) + O(log(n))</i>,
+ assuming that primary keys are accessed equiprobably. 3 and 4
+ are similar to 1, but with lower constants. Using 5 with a
+ hash-based container, the expected cost is <i>O(1)</i>; using 5
+ with a tree based container, the cost is
+ <i><b>E</b>(Θ(log(mn))) = Θ(log(m) +
+ log(n))</i>.</p>
+<p>Suppose one needs the values whose primary key matches some
+ given key. Using 1 with a hash-based container, the expected
+ cost is <i>Θ(n)</i>, but the values will not be ordered
+ by secondary keys (which may or may not be required); using 1
+ with a tree-based container, the expected cost is
+ <i>Θ(log(m) + n)</i>, but with high constants; again the
+ values will not be ordered by secondary keys. 2, 3, and 4 are
+ similar to 1, but typically with lower constants (and,
+ additionally, if one uses a tree-based container for secondary
+ keys, they will be ordered). Using 5 with a hash-based
+ container, the cost is <i>Θ(mn)</i>.</p>
+<p>Suppose one wants to assign to a primary key all secondary
+ keys assigned to a different primary key. Using 1 with a
+ hash-based container, the expected cost is <i>Θ(n)</i>,
+ but with very high constants; using 1 with a tree-based
+ container, the cost is <i>Θ(nlog(mn))</i>. Using 2, 3,
+ and 4, the expected cost is <i>Θ(n)</i>, but typically
+ with far lower costs than 1. 5 is similar to 1.</p>
+</div>
+</body>
+</html>
projects / msp430-gcc.git / blobdiff