跟益达学Solr5之solrconfig.xml配置详解
solrconfig.xml配置文件中包含了很多solr自身配置相关的参数,solrconfig.xml配置文件示例可以从solr的解压目录下找到,如图:
用文本编辑软件打开solrconfig.xml配置,你将会看到以下配置内容:
<?xml version="1.0" encoding="UTF-8" ?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <!-- For more details about configurations options that may appear in this file, see http://wiki.apache.org/solr/SolrConfigXml. --> <config> <!-- In all configuration below, a prefix of "solr." for class names is an alias that causes solr to search appropriate packages, including org.apache.solr.(search|update|request|core|analysis) You may also specify a fully qualified Java classname if you have your own custom plugins. --> <!-- Controls what version of Lucene various components of Solr adhere to. Generally, you want to use the latest version to get all bug fixes and improvements. It is highly recommended that you fully re-index after changing this setting as it can affect both how text is indexed and queried. --> <luceneMatchVersion>5.1.0</luceneMatchVersion> <!-- Data Directory Used to specify an alternate directory to hold all index data other than the default ./data under the Solr home. If replication is in use, this should match the replication configuration. --> <!-- <dataDir>${solr.data.dir:}</dataDir> --> <dataDir>C:\solr_home\core1\data</dataDir> <!-- The DirectoryFactory to use for indexes. solr.StandardDirectoryFactory is filesystem based and tries to pick the best implementation for the current JVM and platform. solr.NRTCachingDirectoryFactory, the default, wraps solr.StandardDirectoryFactory and caches small files in memory for better NRT performance. One can force a particular implementation via solr.MMapDirectoryFactory, solr.NIOFSDirectoryFactory, or solr.SimpleFSDirectoryFactory. solr.RAMDirectoryFactory is memory based, not persistent, and doesn't work with replication. --> <directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"> </directoryFactory> <!-- The CodecFactory for defining the format of the inverted index. The default implementation is SchemaCodecFactory, which is the official Lucene index format, but hooks into the schema to provide per-field customization of the postings lists and per-document values in the fieldType element (postingsFormat/docValuesFormat). Note that most of the alternative implementations are experimental, so if you choose to customize the index format, it's a good idea to convert back to the official format e.g. via IndexWriter.addIndexes(IndexReader) before upgrading to a newer version to avoid unnecessary reindexing. --> <codecFactory class="solr.SchemaCodecFactory"/> <schemaFactory class="ClassicIndexSchemaFactory"/> <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Index Config - These settings control low-level behavior of indexing Most example settings here show the default value, but are commented out, to more easily see where customizations have been made. Note: This replaces <indexDefaults> and <mainIndex> from older versions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --> <indexConfig> <!-- LockFactory This option specifies which Lucene LockFactory implementation to use. single = SingleInstanceLockFactory - suggested for a read-only index or when there is no possibility of another process trying to modify the index. native = NativeFSLockFactory - uses OS native file locking. Do not use when multiple solr webapps in the same JVM are attempting to share a single index. simple = SimpleFSLockFactory - uses a plain file for locking Defaults: 'native' is default for Solr3.6 and later, otherwise 'simple' is the default More details on the nuances of each LockFactory... http://wiki.apache.org/lucene-java/AvailableLockFactories --> <lockType>${solr.lock.type:native}</lockType> <!-- Lucene Infostream To aid in advanced debugging, Lucene provides an "InfoStream" of detailed information when indexing. Setting the value to true will instruct the underlying Lucene IndexWriter to write its info stream to solr's log. By default, this is enabled here, and controlled through log4j.properties. --> <infoStream>true</infoStream> </indexConfig> <!-- JMX This example enables JMX if and only if an existing MBeanServer is found, use this if you want to configure JMX through JVM parameters. Remove this to disable exposing Solr configuration and statistics to JMX. For more details see http://wiki.apache.org/solr/SolrJmx --> <jmx /> <!-- If you want to connect to a particular server, specify the agentId --> <!-- <jmx agentId="myAgent" /> --> <!-- If you want to start a new MBeanServer, specify the serviceUrl --> <!-- <jmx serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/> --> <!-- The default high-performance update handler --> <updateHandler class="solr.DirectUpdateHandler2"> <!-- Enables a transaction log, used for real-time get, durability, and and solr cloud replica recovery. The log can grow as big as uncommitted changes to the index, so use of a hard autoCommit is recommended (see below). "dir" - the target directory for transaction logs, defaults to the solr data directory. --> <updateLog> <str name="dir">${solr.ulog.dir:}</str> </updateLog> <!-- AutoCommit Perform a hard commit automatically under certain conditions. Instead of enabling autoCommit, consider using "commitWithin" when adding documents. http://wiki.apache.org/solr/UpdateXmlMessages maxDocs - Maximum number of documents to add since the last commit before automatically triggering a new commit. maxTime - Maximum amount of time in ms that is allowed to pass since a document was added before automatically triggering a new commit. openSearcher - if false, the commit causes recent index changes to be flushed to stable storage, but does not cause a new searcher to be opened to make those changes visible. If the updateLog is enabled, then it's highly recommended to have some sort of hard autoCommit to limit the log size. --> <autoCommit> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> <openSearcher>false</openSearcher> </autoCommit> <!-- softAutoCommit is like autoCommit except it causes a 'soft' commit which only ensures that changes are visible but does not ensure that data is synced to disk. This is faster and more near-realtime friendly than a hard commit. --> <autoSoftCommit> <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime> </autoSoftCommit> </updateHandler> <!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Query section - these settings control query time things like caches ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --> <query> <!-- Max Boolean Clauses Maximum number of clauses in each BooleanQuery, an exception is thrown if exceeded. ** WARNING ** This option actually modifies a global Lucene property that will affect all SolrCores. If multiple solrconfig.xml files disagree on this property, the value at any given moment will be based on the last SolrCore to be initialized. --> <maxBooleanClauses>1024</maxBooleanClauses> <!-- Solr Internal Query Caches There are two implementations of cache available for Solr, LRUCache, based on a synchronized LinkedHashMap, and FastLRUCache, based on a ConcurrentHashMap. FastLRUCache has faster gets and slower puts in single threaded operation and thus is generally faster than LRUCache when the hit ratio of the cache is high (> 75%), and may be faster under other scenarios on multi-cpu systems. --> <!-- Filter Cache Cache used by SolrIndexSearcher for filters (DocSets), unordered sets of *all* documents that match a query. When a new searcher is opened, its caches may be prepopulated or "autowarmed" using data from caches in the old searcher. autowarmCount is the number of items to prepopulate. For LRUCache, the autowarmed items will be the most recently accessed items. Parameters: class - the SolrCache implementation LRUCache or (LRUCache or FastLRUCache) size - the maximum number of entries in the cache initialSize - the initial capacity (number of entries) of the cache. (see java.util.HashMap) autowarmCount - the number of entries to prepopulate from and old cache. --> <filterCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"/> <!-- Query Result Cache Caches results of searches - ordered lists of document ids (DocList) based on a query, a sort, and the range of documents requested. --> <queryResultCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/> <!-- Document Cache Caches Lucene Document objects (the stored fields for each document). Since Lucene internal document ids are transient, this cache will not be autowarmed. --> <documentCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/> <!-- custom cache currently used by block join --> <cache name="perSegFilter" class="solr.search.LRUCache" size="10" initialSize="0" autowarmCount="10" regenerator="solr.NoOpRegenerator" /> <!-- Lazy Field Loading If true, stored fields that are not requested will be loaded lazily. This can result in a significant speed improvement if the usual case is to not load all stored fields, especially if the skipped fields are large compressed text fields. --> <enableLazyFieldLoading>true</enableLazyFieldLoading> <!-- Result Window Size An optimization for use with the queryResultCache. When a search is requested, a superset of the requested number of document ids are collected. For example, if a search for a particular query requests matching documents 10 through 19, and queryWindowSize is 50, then documents 0 through 49 will be collected and cached. Any further requests in that range can be satisfied via the cache. --> <queryResultWindowSize>20</queryResultWindowSize> <!-- Maximum number of documents to cache for any entry in the queryResultCache. --> <queryResultMaxDocsCached>200</queryResultMaxDocsCached> <!-- Use Cold Searcher If a search request comes in and there is no current registered searcher, then immediately register the still warming searcher and use it. If "false" then all requests will block until the first searcher is done warming. --> <useColdSearcher>false</useColdSearcher> <!-- Max Warming Searchers Maximum number of searchers that may be warming in the background concurrently. An error is returned if this limit is exceeded. Recommend values of 1-2 for read-only slaves, higher for masters w/o cache warming. --> <maxWarmingSearchers>2</maxWarmingSearchers> </query> <!-- Request Dispatcher This section contains instructions for how the SolrDispatchFilter should behave when processing requests for this SolrCore. handleSelect is a legacy option that affects the behavior of requests such as /select?qt=XXX handleSelect="true" will cause the SolrDispatchFilter to process the request and dispatch the query to a handler specified by the "qt" param, assuming "/select" isn't already registered. handleSelect="false" will cause the SolrDispatchFilter to ignore "/select" requests, resulting in a 404 unless a handler is explicitly registered with the name "/select" handleSelect="true" is not recommended for new users, but is the default for backwards compatibility --> <requestDispatcher handleSelect="false" > <!-- Request Parsing These settings indicate how Solr Requests may be parsed, and what restrictions may be placed on the ContentStreams from those requests enableRemoteStreaming - enables use of the stream.file and stream.url parameters for specifying remote streams. multipartUploadLimitInKB - specifies the max size (in KiB) of Multipart File Uploads that Solr will allow in a Request. formdataUploadLimitInKB - specifies the max size (in KiB) of form data (application/x-www-form-urlencoded) sent via POST. You can use POST to pass request parameters not fitting into the URL. addHttpRequestToContext - if set to true, it will instruct the requestParsers to include the original HttpServletRequest object in the context map of the SolrQueryRequest under the key "httpRequest". It will not be used by any of the existing Solr components, but may be useful when developing custom plugins. *** WARNING *** The settings below authorize Solr to fetch remote files, You should make sure your system has some authentication before using enableRemoteStreaming="true" --> <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="2048000" formdataUploadLimitInKB="2048" addHttpRequestToContext="false"/> <!-- HTTP Caching Set HTTP caching related parameters (for proxy caches and clients). The options below instruct Solr not to output any HTTP Caching related headers --> <httpCaching never304="true" /> </requestDispatcher> <!-- Request Handlers http://wiki.apache.org/solr/SolrRequestHandler Incoming queries will be dispatched to a specific handler by name based on the path specified in the request. Legacy behavior: If the request path uses "/select" but no Request Handler has that name, and if handleSelect="true" has been specified in the requestDispatcher, then the Request Handler is dispatched based on the qt parameter. Handlers without a leading '/' are accessed this way like so: http://host/app/[core/]select?qt=name If no qt is given, then the requestHandler that declares default="true" will be used or the one named "standard". If a Request Handler is declared with startup="lazy", then it will not be initialized until the first request that uses it. --> <!-- SearchHandler http://wiki.apache.org/solr/SearchHandler For processing Search Queries, the primary Request Handler provided with Solr is "SearchHandler" It delegates to a sequent of SearchComponents (see below) and supports distributed queries across multiple shards --> <!-- <requestHandler name="/dataimport" class="solr.DataImportHandler"> <lst name="defaults"> <str name="config">solr-data-config.xml</str> </lst> </requestHandler> --> <requestHandler name="/dataimport" class="solr.DataImportHandler"> <lst name="defaults"> <str name="config">data-config.xml</str> </lst> </requestHandler> <requestHandler name="/select" class="solr.SearchHandler"> <!-- default values for query parameters can be specified, these will be overridden by parameters in the request --> <lst name="defaults"> <str name="echoParams">explicit</str> <int name="rows">10</int> </lst> </requestHandler> <!-- A request handler that returns indented JSON by default --> <requestHandler name="/query" class="solr.SearchHandler"> <lst name="defaults"> <str name="echoParams">explicit</str> <str name="wt">json</str> <str name="indent">true</str> <str name="df">text</str> </lst> </requestHandler> <!-- The export request handler is used to export full sorted result sets. Do not change these defaults. --> <requestHandler name="/export" class="solr.SearchHandler"> <lst name="invariants"> <str name="rq">{!xport}</str> <str name="wt">xsort</str> <str name="distrib">false</str> </lst> <arr name="components"> <str>query</str> </arr> </requestHandler> <initParams path="/update/**,/query,/select,/tvrh,/elevate,/spell"> <lst name="defaults"> <str name="df">text</str> </lst> </initParams> <!-- Field Analysis Request Handler RequestHandler that provides much the same functionality as analysis.jsp. Provides the ability to specify multiple field types and field names in the same request and outputs index-time and query-time analysis for each of them. Request parameters are: analysis.fieldname - field name whose analyzers are to be used analysis.fieldtype - field type whose analyzers are to be used analysis.fieldvalue - text for index-time analysis q (or analysis.q) - text for query time analysis analysis.showmatch (true|false) - When set to true and when query analysis is performed, the produced tokens of the field value analysis will be marked as "matched" for every token that is produces by the query analysis --> <requestHandler name="/analysis/field" startup="lazy" class="solr.FieldAnalysisRequestHandler" /> <!-- Document Analysis Handler http://wiki.apache.org/solr/AnalysisRequestHandler An analysis handler that provides a breakdown of the analysis process of provided documents. This handler expects a (single) content stream with the following format: <docs> <doc> <field name="id">1</field> <field name="name">The Name</field> <field name="text">The Text Value</field> </doc> <doc>...</doc> <doc>...</doc> ... </docs> Note: Each document must contain a field which serves as the unique key. This key is used in the returned response to associate an analysis breakdown to the analyzed document. Like the FieldAnalysisRequestHandler, this handler also supports query analysis by sending either an "analysis.query" or "q" request parameter that holds the query text to be analyzed. It also supports the "analysis.showmatch" parameter which when set to true, all field tokens that match the query tokens will be marked as a "match". --> <requestHandler name="/analysis/document" class="solr.DocumentAnalysisRequestHandler" startup="lazy" /> <!-- Echo the request contents back to the client --> <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" > <lst name="defaults"> <str name="echoParams">explicit</str> <str name="echoHandler">true</str> </lst> </requestHandler> <!-- Search Components Search components are registered to SolrCore and used by instances of SearchHandler (which can access them by name) By default, the following components are available: <searchComponent name="query" class="solr.QueryComponent" /> <searchComponent name="facet" class="solr.FacetComponent" /> <searchComponent name="mlt" class="solr.MoreLikeThisComponent" /> <searchComponent name="highlight" class="solr.HighlightComponent" /> <searchComponent name="stats" class="solr.StatsComponent" /> <searchComponent name="debug" class="solr.DebugComponent" /> --> <!-- Terms Component http://wiki.apache.org/solr/TermsComponent A component to return terms and document frequency of those terms --> <searchComponent name="terms" class="solr.TermsComponent"/> <!-- A request handler for demonstrating the terms component --> <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy"> <lst name="defaults"> <bool name="terms">true</bool> <bool name="distrib">false</bool> </lst> <arr name="components"> <str>terms</str> </arr> </requestHandler> <!-- Legacy config for the admin interface --> <admin> <defaultQuery>*:*</defaultQuery> </admin> </config>
下面我将对其中关键地方加以解释说明:
lib
<lib> 标签指令可以用来告诉Solr如何去加载solr plugins(Solr插件)依赖的jar包,在solrconfig.xml配置文件的注释中有配置示例,例如:
<lib dir="./lib" regex=”lucene-\w+\.jar”/>
这里的dir表示一个jar包目录路径,该目录路径是相对于你当前core根目录的;regex表示一个正则表达式,用来过滤文件名的,符合正则表达式的jar文件将会被加载
dataDir parameter
用来指定一个solr的索引数据目录,solr创建的索引会存放在data\index目录下,默认dataDir是相对于当前core目录(如果solr_home下存在core的话),如果solr_home下不存在core的话,那dataDir默认就是相对于solr_home啦,不过一般dataDir都在core.properties下配置。
<dataDir>/var/data/solr</dataDir>
codecFactory
用来设置Lucene倒排索引的编码工厂类,默认实现是官方提供的SchemaCodecFactory类。
indexConfig Section
在solrconfig.xml的<indexConfig>标签中间有很多关于此配置项的说明:
<!-- maxFieldLength was removed in 4.0. To get similar behavior, include a
LimitTokenCountFilterFactory in your fieldType definition. E.g.
<filter class="solr.LimitTokenCountFilterFactory" maxTokenCount="10000"/>
提供我们maxFieldLength配置项已经从4.0版本开始就已经被移除了,可以使用配置一个filter达到相似的效果,maxTokenCount即在对某个域分词的时候,最多只提取前10000个Token,后续的域值将被抛弃。maxFieldLength若表示1000,则意味着只会对域值的0~1000范围内的字符串进行分词索引。
<writeLockTimeout>1000</writeLockTimeout>
writeLockTimeout表示IndexWriter实例在获取写锁的时候最大等待超时时间,超过指定的超时时间仍未获取到写锁,则IndexWriter写索引操作将会抛出异常
<maxIndexingThreads>8</maxIndexingThreads>
表示创建索引的最大线程数,默认是开辟8个线程来创建索引
<useCompoundFile>false</useCompoundFile>
是否开启复合文件模式,启用了复合文件模式即意味着创建的索引文件数量会减少,这样占用的文件描述符也会减少,但这会带来性能的损耗,在Lucene中,它默认是开启,而在Solr中,自从3.6版本开始,默认就是禁用的
<ramBufferSizeMB>100</ramBufferSizeMB>
表示创建索引时内存缓存大小,单位是MB,默认最大是100M,
<maxBufferedDocs>1000</maxBufferedDocs>
表示在document写入到硬盘之前,缓存的document最大个数,超过这个最大值会触发索引的flush操作。
<mergePolicy class="org.apache.lucene.index.TieredMergePolicy"> <int name="maxMergeAtOnce">10</int> <int name="segmentsPerTier">10</int> </mergePolicy>
用来配置Lucene索引段合并策略的,里面有两个参数:
maxMergeAtOne: 一次最多合并段个数
segmentPerTier: 每个层级的段个数,同时也是内存buffer递减的等比数列的公比,看源码:
// Compute max allowed segs in the index long levelSize = minSegmentBytes; long bytesLeft = totIndexBytes; double allowedSegCount = 0; while(true) { final double segCountLevel = bytesLeft / (double) levelSize; if (segCountLevel < segsPerTier) { allowedSegCount += Math.ceil(segCountLevel); break; } allowedSegCount += segsPerTier; bytesLeft -= segsPerTier * levelSize; levelSize *= maxMergeAtOnce; } int allowedSegCountInt = (int) allowedSegCount;
<mergeFactor>10</mergeFactor>
要理解mergeFactor因子的含义,还是先看看lucene in action中给出的解释:
IndexWriter’s mergeFactor lets you control how many Documents to store in memory before writing them to the disk, as well as how often to merge multiple index segments together. (Index segments are covered in appendix B.) With the default value of 10, Lucene stores 10 Documents in memory before writing them to a single segment on the disk. The mergeFactor value of 10 also means that once the number of segments on the disk has reached the power of 10, Lucene merges these segments into a single segment. For instance, if you set mergeFactor to 10, a new segment is created on the disk for every 10 Documents added to the index. When the tenth segment of size 10 is added, all 10 are merged into a single segment of size 100. When 10 such segments of size 100 have been added, they’re merged into a single segment containing 1,000 Documents, and so on. Therefore, at any time, there are no more than 9 segments in the index, and the size of each merged segment is the power of 10. There is a small exception to this rule that has to do with maxMergeDocs, another IndexWriter instance variable: While merging segments, Lucene ensuresthat no segment with more than maxMergeDocs Documents is created. For instance, suppose you set maxMergeDocs to 1,000. When you add the ten-thousandth Document, instead of merging multiple segments into a single segment of size 10,000, Lucene creates the tenth segment of size 1,000 and keeps adding new segments of size 1,000 for every 1,000 Documents added.
IndexWriter的mergeFactory允许你来控制索引在写入磁盘之前内存中能缓存的document数量,以及合并
多个段文件的频率。默认这个值为10. 当往内存中存储了10个document,此时Lucene还没有把单个段文件
写入磁盘,mergeFactor值等于10也意味着当硬盘上的段文件数量达到10,lucene将会把这10个段文件合
并到一个段文件中。例如:如果你把mergeFactor设置为10,当你往索引中添加了10个document,一个段
文件将会在硬盘上被创建,当第10个段文件被添加时,这10个段文件就会被合并到1个段文件,此时这个
段文件中有100个document,当10个这样的包含了100个document的段文件被添加时,他们又会被合并到一
个新的段文件中,而此时这个段文件包含 1000个document,以此类推。所以,在任何时候,在索引中不
存在超过9个段文件。每个被合并的段文件包含的document个数都是10,但这样有点小问题,我们还必须
设置一个maxMergeDocs变量,当合并段文件的时候,lucene必须确保没有哪个段文件超过maxMergeDocs
变量规定的最大document数量。设置maxMergeDocs的目的是为了防止单个段文件中包含的document数量
过大,假定你把maxMergeDocs设置为1000,当你创建第10个包含1000个document段文件的时候,这时并
不会触发段文件合并(如果没有设置maxMergeDocs为100的话,按理来说,这10个包含了1000个document
的段文件将会被合并到一个包含了10000个document的段文件当中,但maxMergeDocs限制了单个段文件中
最多包含1000个document,所以此时并不会触发段合并操作)。影响段合并还有一些其他参数,比如:
mergeFactor:当大小几乎相当的段的数量达到此值的时候,开始合并。
minMergeSize:所有大小小于此值的段,都被认为是大小几乎相当,一同参与合并。
maxMergeSize:当一个段的大小大于此值的时候,就不再参与合并。
maxMergeDocs:当一个段包含的文档数大于此值的时候,就不再参与合并。
段合并分两个步骤:
1.首先筛选出哪些段需要合并,这一步由MergePolicy合并策略类来决定
2.然后就是真正的段合并过程了,这一步是交给MergeScheduler来完成的,MergeScheduler类主要做两件事:
A.对存储域,项向量,标准化因子即norms等信息进行合并
B.对倒排索引信息进行合并
尼玛扯远了,接着继续我们的solrconfig.xml中影响索引创建的一些参数配置;
<mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
mergeScheduler刚才提到过了,这是用来配置段合并操作的处理类。默认实现类是Lucene中自带的ConcurrentMergeScheduler。
<lockType>${solr.lock.type:native}</lockType>
这个是用来指定Lucene中LockFactory实现的,可配置项如下:
single = SingleInstanceLockFactory - suggested for a read-only index or when there is no possibility of another process trying to modify the index. native = NativeFSLockFactory - uses OS native file locking. Do not use when multiple solr webapps in the same JVM are attempting to share a single index. simple = SimpleFSLockFactory - uses a plain file for locking Defaults: 'native' is default for Solr3.6 and later, otherwise 'simple' is the default
single:表示只读锁,没有另外一个处理线程会去修改索引数据
native:即Lucene中的NativeFSLockFactory实现,使用的是基于操作系统的本地文件锁
simple:即Lucene中的SimpleFSLockFactory实现,通过在硬盘上创建write.lock锁文件实现
Defaults:从solr3.6版本开始,这个默认值是native,否则,默认值就是simple,意思就是说,你如果配置为Defaults,到底使用哪种锁实现,取决于你当前使用的Solr版本。
<unlockOnStartup>false</unlockOnStartup>
如果这个设置为true,那么在solr启动后,IndexWriter和commit提交操作拥有的锁将会被释放,这会打破Lucene的锁机制,请谨慎使用。如果你的lockType设置为single,那么这个配置true or false都不会产生任何影响。
<deletionPolicy class="solr.SolrDeletionPolicy">
用来配置索引删除策略的,默认使用的是Solr的SolrDeletionPolicy实现。如果你需要自定义删除策略,那么你需要实现Lucene的org.apache.lucene.index.IndexDeletionPolicy接口。
<jmx />
这个配置是用来在Solr中启用JMX,有关这方面的详细信息,请移步到Solr官方Wiki,访问地址如下:
http://wiki.apache.org/solr/SolrJmx
<updateHandler class="solr.DirectUpdateHandler2">
指定索引更新操作处理类,DirectUpdateHandler2是一个高性能的索引更新处理类,它支持软提交
<updateLog>
<str name="dir">${solr.ulog.dir:}</str>
</updateLog>
<updateLog>用来指定上面的updateHandler的处理事务日志存放路径的,默认值是solr的data目录即solr的dataDir配置的目录。
<query>标签是有关索引查询相关的配置项
<maxBooleanClauses>1024</maxBooleanClauses>
表示BooleanQuery最大能链接多少个子Query,当不同的core下的solrconfig.xml中此配置项的参数值配置的不一样时,以最后一个初始化的core的配置为准。
<filterCache class="solr.FastLRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
用来配置filter过滤器的缓存相关的参数
<queryResultCache class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
用来配置对Query返回的查询结果集即TopDocs的缓存
<documentCache class="solr.LRUCache"
size="512"
initialSize="512"
autowarmCount="0"/>
用来配置对Document中存储域的缓存,因为每次从硬盘上加载存储域的值都是很昂贵的操作,这里说的存储域指的是那些Store.YES的Field,所以你懂的。
<fieldValueCache class="solr.FastLRUCache"
size="512"
autowarmCount="128"
showItems="32" />
这个配置是用来缓存Document id的,用来快速访问你的Document id的。这个配置项默认就是开启的,无需显式配置。
<cache name="myUserCache"
class="solr.LRUCache"
size="4096"
initialSize="1024"
autowarmCount="1024"
regenerator="com.mycompany.MyRegenerator"
/>
这个配置是用来配置你的自定义缓存的,你自己的Regenerator需要实现Solr的CacheRegenerator接口。
<enableLazyFieldLoading>true</enableLazyFieldLoading>
表示启用存储域的延迟加载,前提是你的存储域在Query的时候没有显式指定需要return这个域。
<useFilterForSortedQuery>true</useFilterForSortedQuery>
表示当你的Query没有使用score进行排序时,是否使用filter来替代Query.
<listener event="newSearcher" class="solr.QuerySenderListener"> <arr name="queries"> <!-- <lst><str name="q">solr</str><str name="sort">price asc</str></lst> <lst><str name="q">rocks</str><str name="sort">weight asc</str></lst> --> </arr> </listener>
QuerySenderListener用来监听查询发送过程,即你可以在Query请求发送之前追加一些请求参数,如上面给的示例中,可以追加qery关键字以及sort排序规则。
<requestDispatcher handleSelect="false" >
设置为false即表示Solr 服务器端不接收/select请求,即如果你请求http://localhost:8080/solr/coreName/select?qt=xxxx时,将会返回一个404,
这个select请求是为了兼容先前的旧版本,已经不推荐使用。
<httpCaching never304="true" />
表示solr服务器段永远不返回304,那http响应状态码304表示什么呢?表示服务器端告诉客户端,你请求的资源尚未被修改过,我返回给你的是上次缓存的内容。Never304即告诉服务器,不管我访问的资源有没有更新过,都给我重新返回不走Http缓存。这属于Http协议相关知识,不清楚的请去Google HTTP协议详细了解去。
<requestHandler name="/query" class="solr.SearchHandler"> <lst name="defaults"> <str name="echoParams">explicit</str> <str name="wt">json</str> <str name="indent">true</str> </lst> </requestHandler>
这个requestHandler配置的是请求URL /query跟请求处理类SearcherHandler之间的一个映射关系,即你访问http://localhost:8080/solr/coreName/query?q=xxx时,会交给SearcherHandler类来处理这个http请求,你可以配置一些参数来干预SearcherHandler处理细节,比如echoParams表示是否打印HTTP请求参数,wt即writer type,即返回的数据的MIME类型,如json,xml等等,indent表示返回的json或者XML数据是否需要缩进,否则返回的数据没有缩进也没有换行,不利于阅读。
其他的一些requestHandler说明就略过了,其实都大同小异,就是一个请求URL跟请求处理类的一个映射,就好比SpringMVC中请求URL和Controller类的一个映射。
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
用来配置查询组件比如SpellCheckComponent拼写检查,有关拼写检查的详细配置说明留到以后说到SpellCheck时再说吧。
<searchComponent name="terms" class="solr.TermsComponent"/>
用来返回所有的Term以及每个document中Term的出现频率
<searchComponent class="solr.HighlightComponent" name="highlight">
用来配置关键字高亮的,Solr高亮配置的详细说明这里暂时先略过,这篇我们只是先暂时大致了解下每个配置项的含义即可,具体如何使用留到后续再深入研究。
有关searchComponent查询组件的其他配置我就不一一说明了,太多了。你们自己看里面的英文注释吧,如果你实在看不懂再来问我。
<queryResponseWriter name="json" class="solr.JSONResponseWriter"> <!-- For the purposes of the tutorial, JSON responses are written as plain text so that they are easy to read in *any* browser. If you expect a MIME type of "application/json" just remove this override. --> <str name="content-type">text/plain; charset=UTF-8</str> </queryResponseWriter>
这个是用来配置Solr响应数据转换类,JSONResponseWriter就是把HTTP响应数据转成JSON格式,content-type即response响应头信息中的content-type,即告诉客户端返回的数据的MIME类型为text/plain,且charset字符集编码为UTF-8.
内置的响应数据转换器还有velocity,xslt等,如果你想自定义一个基于FreeMarker的转换器,那你需要实现Solr的QueryResponseWriter接口,模仿其他实现类,你懂的,然后在solrconfig.xml中添加类似的<queryResponseWriter配置即可
最后需要说明下的是solrconfig.xml中有大量类似<arr> <list> <str> <int>这样的自定义标签,下面做个统一的说明:
这张图摘自于Solr in Action这本书,由于是英文的,所以我稍微解释下:
arr:即array的缩写,表示一个数组,name即表示这个数组参数的变量名
lst即list的缩写,但注意它里面存放的是key-value键值对
bool表示一个boolean类型的变量,name表示boolean变量名,
同理还有int,long,float,str等等
Str即string的缩写,唯一要注意的是arr下的str子元素是没有name属性的,而list下的str元素是有name属性的
最后总结下:
solrconfig.xml中的配置项主要分以下几大块:
1.依赖的lucene版本配置,这决定了你创建的Lucene索引结构,因为Lucene各版本之间的索引结构并不是完全兼容的,这个需要引起你的注意。
2.索引创建相关的配置,如索引目录,IndexWriterConfig类中的相关配置(它决定了你的索引创建性能)
3.solrconfig.xml中依赖的外部jar包加载路径配置
4.JMX相关配置
5.缓存相关配置,缓存包括过滤器缓存,查询结果集缓存,Document缓存,以及自定义缓存等等
6.updateHandler配置即索引更新操作相关配置
7.RequestHandler相关配置,即接收客户端HTTP请求的处理类配置
8.查询组件配置如HightLight,SpellChecker等等
9.ResponseWriter配置即响应数据转换器相关配置,决定了响应数据是以什么样格式返回给客户端的。
10.自定义ValueSourceParser配置,用来干预Document的权重、评分,排序
solrconfig.xml就解释到这儿了,理解这些配置项是为后续Solr学习扫清障碍。有些我没说到的或者我有意略过的,就留给你们自己去阅读和理解了,毕竟内容太多,1000多行的配置,一行不拉的解释完太耗时,有些都是类似的配置,我想你们应该能看懂。
如果你还有什么问题请加我Q-Q:7-3-6-0-3-1-3-0-5,
或者加裙
一起交流学习!