Step-by-Step: Provisioning the Search Service Application

Provisioning the Search Service Application
Open SharePoint 2010 Central Administration.
Select Managed service applications under Application Management.
Select New | Search Service Application on the ribbon user interface.

On the Create Search Service Application dialog specify the name for the new Search Service Application or accept the default name, usually Search Service Application 1.
Provide a name for the new Search Administration Web Service Application Pool or use an existing Application Pool.
Provide a name for the new Search Administration Site Settings and Query Web Service or use an existing Application Pool.

Click OK on the new Create New Search Service Application dialog to provision the new service application
Once the Search Service Application has been successfully provisioned on the server farm you will have a 1x1x1 topology or otherwise 1 Search Administration, 1 Crawl, and 1 Query component on the machine hosting SharePoint 2010 Central Administration and all associated databases on the default database server.

NOTES
The Search administration (Admin) topology does not scale out - there can be on one (1) search administration component and one (1) search administration database per Search Service Application.
The Crawl topology can be scaled out by adding Crawl Components or Crawl Databases.  Crawl Components can have a many-to-one relationships with Crawl Databases.
The Query topology can be scaled out by adding Property Databases or by adding Query Components.  Index Partitions subdivide the full-text index.   A new Query Component can either be the first component in a new partition (see above illustration (Query Component 0)) or an additional component in an existing partition.
In the public beta, Index Partitions have a many-to-one relationship with Property Databases.
Moving Query Components
Open SharePoint 2010 Central Administration.
Select Managed service applications under Application Management.
On the Services Applications page, select the Search Service Application.
On the Search Administration page, locate the Search Application Topology section and click Modify.
On the Topology for Search Service Application: Search Service Application page, locate the Index Partition category. (The default Query Component is typically named Query Component 0). Click Query Component 0 and then click Edit Properties.
On the Edit Query Component page, select a server in the topology from the Server drop-down list and then click OK.  This will move the Query Component to the selected server.

Creating Mirror Query Components
When you create a Mirror Query Component, you create a replica of the Index Partition on another server.  You will typically create new Mirror Query Components when you need to increase throughput or availability.
Open SharePoint 2010 Central Administration.
Select Managed service applications under Application Management.
On the Services Applications page, select the Search Service Application.
On the Search Administration page, locate the Search Application Topology section and click Modify.
On the Topology for Search Service Application: Search Service Application page, locate the Index Partition category. (The default Query Component is typically named Query Component 0). Click Query Component 0 and then click Add Mirror.

On the Add mirror query component dialog, select a server in the topology from the Server drop-down list and then click OK.

Repeat the steps for each server in the topology as required.
Creating Query Components
When you create a new Query Component, you create a new Index Partition which subdivides the full-text index.  You will typically create new Query Components and Index Partitions when the total number of items in your Index exceed the recommend scale for a single Index Partition, or when you need to increase throughput or availability.
Open SharePoint 2010 Central Administration.
Select Managed service applications under Application Management.
On the Services Applications page, select the Search Service Application.
On the Search Administration page, locate the Search Application Topology section and click Modify.
On the Topology for Search Service Application:  Search Service Application 1, select New | Index Partition and Query Component.

On the Add Query Component dialog, select a server in the topology from the Server drop-down list, Property Database, and specify the location of the Index Partition.

Click OK on the Add Query Component dialog to save the changes and create the new Query Component.
Creating Crawl Components
You will typically create new Crawl Components to improve the overall crawl speed and subsequently freshness of the content and to improve availability.
Open SharePoint 2010 Central Administration.
Select Managed service applications under Application Management.
On the Services Applications page, select the Search Service Application.
On the Search Administration page, locate the Search Application Topology section and click Modify.
On the Topology for Search Service Application:  Search Service Application 1, select New | Crawl Component.

On the Add Crawl Component dialog specify the server where the Crawl Component will be hosted, the Crawl Database to which the Crawl Component will be associated, and the temporary location on the Index.

Click OK on the Add Crawl Component dialog to save the changes and create the new Crawl Component.
Creating Crawl Databases
You will typically create new Crawl Databases to improve the overall crawl speed and subsequently freshness of the content and in correlation to the creation of new Crawl Components.
Open SharePoint 2010 Central Administration.
Select Managed service applications under Application Management.
On the Services Applications page, select the Search Service Application.
On the Search Administration page, locate the Search Application Topology section and click Modify.
On the Topology for Search Service Application:  Search Service Application 1, select New | Crawl Database.

On the Add Crawl Database dialog specify the database server where the Crawl Database will reside, the database name, and optionally the select whether the Crawl Database will be dedicated to hosts specified in Host Distribution Rules.
Host Distribution Rules are useful in specifying:
1. A particular host that is processed by a one or more Crawler Databases.
2. A particular host is processed by only one or more Crawler Database.
Host Distribution Rules are commonly used to support large and complex content corpuses that require horizontal scale (scale out) topologies.

Click OK on the Add Crawl Database dialog to save the changes and create the new Crawl Database.
Creating Property Databases
You will typically create new Property Databases to support the horizontal scale (scale out) of the Query Component(s).
Open SharePoint 2010 Central Administration.
Select Managed service applications under Application Management.
On the Services Applications page, select the Search Service Application.
On the Search Administration page, locate the Search Application Topology section and click Modify.
On the Topology for Search Service Application:  Search Service Application 1, select New | Property Database.

On the Add Property Database dialog specify the database server where the Property Database will reside and the database name.

Click OK on the Add Property Database dialog to save the changes and create the new Property Database.

SharePoint 2010 Configuring Search Service Application using PowerShell

It might be necessary at some point to use PowerShell to provision search service applications.  For Example, setting up a search service application for hosted sites requires you to use PowerShell.  The following steps manually take you through this process and I highly recommend going through the steps to become more familiar with the command-lets.  ​ A sample powershell script is provided at the bottom of this blog. 

Creating Search Service Application using PowerShell

1. Create Application Pool
Creating a an application pool for your search service application and throwing the object into a variable called $ app:
      $app = new-spserviceapplicationpool –name contososearch-apppool –account domain\user

2. Create search service application

      $searchapp = new-spenterprisesearchserviceapplication -name ContosoSearchServiceApplication -applicationpool $app
Note: Add the -partitioned switch after -name if the search service application will be consumed in a hosted environment.                        

3. Create search service application proxy
$proxy = new-spenterprisesearchserviceapplicationproxy -name Contososearchserviceapplicationproxy -Uri $searchapp.uri.absoluteURI
            Note: Add the -partitioned switch if the search service application will be consumed in a hosted environment.

Verify the search service application proxy is online.  It should be online by default..
$proxy.status
If it's not online, you can change the status by punching in the following:

To change this property you could type something like this:
$proxy.status = “online”
Finally, you must update the change by calling the update method.
$changestatus.update()


4.  Ensure the local search service instance is started
Run the following: 
$si = get-spenterprisesearchserviceinstance –local
$si.status
If it's enabled/started, skip to step 5!
If it's disabled then run the following:
Start-SpEnterpriseSearchServiceInstance -identity $SI

5.  Provision Search Administration Component
Configure the administration component of the associated Searchserviceapplication.  You can do this with the following command:
set-spenterprisesearchadministrationcomponent –searchapplication $searchapp  –searchserviceinstance $si

6. Provision Crawl Component and Activate
By default, a search application created in PowerShell has a crawl topology but is missing the following:
· crawl component           
· query component
You cannot add a crawl\query component to the default crawl\query topology because it's set as active and the property is read only.  The easiest way around this is creating a new crawl topology and new query topology.  After creating both, they will be set as inactive by default.  This allows for both crawl components to be added to crawl topology and query component to be added to newly created query topology. Finally, you can set this new crawl topology to active. 

a. Create Crawl Topology

$ct = $searchapp | new-spenterprisesearchcrawltopology

b. Create a new Crawl Store
$csid = $SearchApp.CrawlStores | select id
$CrawlStore = $SearchApp.CrawlStores.item($csid.id)

c. Create a new Crawl Component
Create a crawl component for new crawl topology by passing the variables representing the crawl topology, search instance, and crawlstore.
$hname = hostname
new-spenterprisesearchcrawlcomponent -crawltopology $ct -crawldatabase $Crawlstore -searchserviceinstance $hname

d. Finally, set the new crawl topology as active.
$ct | set-spenterprisesearchcrawltopology -active


7.  Create Query Components and Activate
a. Create a new Query Topology 
$qt = $searchapp | new-spenterprisesearchquerytopology -partitions 1

b. Create a variable for the Query Partition
$p1 = ($qt | get-spenterprisesearchindexpartition)


c. Create a new Query Component
new-spenterprisesearchquerycomponent -indexpartition $p1 -querytopology $qt -searchserviceinstance $si

d. Create a variable for the Property Store DB
$PSID = $SearchApp.PropertyStores | Select id
$PropDB = $SearchApp.PropertyStores.Item($PSID.id)

e. Set the Query Partition to use the Property Store DB
$p1 | set-spenterprisesearchindexpartition -PropertyDatabase $PropDB

f.  Activate the Query Topology
$qt | Set-SPEnterpriseSearchQueryTopology -Active

==========================================================
Sample Script
Thanks is in store to Colin at MSFT for taking the cmdlets above and throwing together a great sample script.   Copy the script below and save it as a .PS1 file.  
Note:  When provisioning a search service application for hosted “multi-tenant” sites, the following cmd-lets must contain the –partitioned parameter.

• New-SPEnterpriseSearchServiceApplication (Step 3 below)
• New-SPEnterpriseSearchServiceApplicationProxy (Step 4 below)

 

Add-PSSnapin Microsoft.SharePoint.PowerShell
# 1.Setting up some initial variables.
write-host 1.Setting up some initial variables.
$SSAName = "ContosoSearch"
$SVCAcct = "Contoso\administrator"
$SSI = get-spenterprisesearchserviceinstance -local
$err = $null
# Start Services search services for SSI
write-host Start Services search services for SSI
Start-SPEnterpriseSearchServiceInstance -Identity $SSI
# 2.Create an Application Pool.
write-host 2.Create an Application Pool.
$AppPool = new-SPServiceApplicationPool -name $SSAName"-AppPool" -account $SVCAcct
# 3.Create the SearchApplication and set it to a variable
write-host 3.Create the SearchApplication and set it to a variable
$SearchApp = New-SPEnterpriseSearchServiceApplication -Name $SSAName -applicationpool $AppPool -databasename $SSAName"_AdminDB"
#4 Create search service application proxy
write-host 4 Create search service application proxy
$SSAProxy = new-spenterprisesearchserviceapplicationproxy -name $SSAName"ApplicationProxy" -Uri $SearchApp.Uri.AbsoluteURI
# 5.Provision Search Admin Component.
write-host 5.Provision Search Admin Component.
set-SPenterprisesearchadministrationcomponent -searchapplication $SearchApp  -searchserviceinstance $SSI
# 6.Create a new Crawl Topology.
write-host 6.Create a new Crawl Topology.
$CrawlTopo = $SearchApp | New-SPEnterpriseSearchCrawlTopology
# 7.Create a new Crawl Store.
write-host 7.Create a new Crawl Store.
$CrawlStore = $SearchApp | Get-SPEnterpriseSearchCrawlDatabase
# 8.Create a new Crawl Component.
write-host 8.Create a new Crawl Component.
New-SPEnterpriseSearchCrawlComponent -CrawlTopology $CrawlTopo -CrawlDatabase $CrawlStore -SearchServiceInstance $SSI
# 9.Activate the Crawl Topology.
write-host 9.Activate the Crawl Topology.
do
{
    $err = $null
    $CrawlTopo | Set-SPEnterpriseSearchCrawlTopology -Active -ErrorVariable err
    if ($CrawlTopo.State -eq "Active")
    {
        $err = $null
    }
    Start-Sleep -Seconds 10
}
until ($err -eq $null)
# 10.Create a new Query Topology.
write-host 10.Create a new Query Topology.
$QueryTopo = $SearchApp | New-SPenterpriseSEarchQueryTopology -partitions 1
# 11.Create a variable for the Query Partition
write-host 11.Create a variable for the Query Partition
$Partition1 = ($QueryTopo | Get-SPEnterpriseSearchIndexPartition)
# 12.Create a Query Component.
write-host 12.Create a Query Component.
New-SPEnterpriseSearchQueryComponent -indexpartition $Partition1 -QueryTopology $QueryTopo -SearchServiceInstance $SSI
# 13.Create a variable for the Property Store DB.
write-host 13.Create a variable for the Property Store DB.
$PropDB = $SearchApp | Get-SPEnterpriseSearchPropertyDatabase
# 14.Set the Query Partition to use the Property Store DB.
write-host 14.Set the Query Partition to use the Property Store DB.
$Partition1 | Set-SPEnterpriseSearchIndexPartition -PropertyDatabase $PropDB
# 15.Activate the Query Topology.
write-host 15.Activate the Query Topology.
do
{
    $err = $null
    $QueryTopo | Set-SPEnterpriseSearchQueryTopology -Active -ErrorVariable err -ErrorAction SilentlyContinue
    Start-Sleep -Seconds 10
    if ($QueryTopo.State -eq "Active")
        {
            $err = $null
        }
}
until ($err -eq $null)
Write-host "Your search application $SSAName is now ready"

SharePoint 2013 incluye FAST for SharePoint como producto integrado!

Con SharePoint 2013 tendremos integradas las características que antes teníamos con el producto separado “FAST for SharePoint 2010”.

Cuales son las novedades principales en este campo. Es más, para hacer uso de las buenas prácticas didácticas, iremos racionalizando el aprendizaje, este será el índice que seguiremos:
*****************************************************************************
1 . Nueva Arquitectura y Topología
2. Novedades de Configuración en Rastreador de Contenido (Crawler Configuration)
      2.1 Conectores (Connectors)
      2.2 Fuentes de contenido y rastreo (Crawling and Content Sources)
      2.3 Fuentes de Resultados (Result Sources)
      2.4 Mejoras en el Parseado de Documentos (Document Parsing)
      2.5 Extracción de Entidades (Entity Extraction)
      2.6 Gestión de Esquemas (Schema Management)
3. Novedades de Configuración en el Motor de Consultas (Query Configuration)
      3.1 Ranking
      3.2 Corrección ortográfica en la consulta (Query Spell Correction)
      3.3 Reglas de consulta (Query Rules)
*****************************************************************************

En este post veremos la nueva Arquitectura y Topología del motor de Búsqueda Empresarial de SharePoint 2013.

Nueva Arquitectura y Topología de SharePoint 2013 Search

Con la integración de características de FAST dentro del motor de búsqueda de SharePoint se cambia totalmente la arquitectura de SharePoint 2013 Search, veamos entonces cada uno de los módulos representados en la siguiente imagen:

 

1. Componente Rastreador (Crawler)

- Este componente es responsable de rastrear el contenido que proviene desde distintas fuentes de información. Invoca a los conectores y manejadores de protocolos adecuados para cada fuente de datos.
- Importante: Podemos desarrollar nuestros propios conectores personalizados Más información (en ingles): http://msdn.microsoft.com/en-us/library/ee556429(v=office.15)
- La base de datos de Crawl (A) se utiliza para almacenar información sobre los elementos rastreados y el historial del rastreo (tiempo del último rastreo, Id del último rastreo, …)

2. Componente de Procesamiento de Contenido (Content Processing)

- Procesa los elementos rastreados y los pasa al componente de Indexación. Aquí es donde se usan los iFilters para hacer el parsing de los documentos. En SharePoint 2013 sigue pudiéndose extender los Format Handlers y crear un propio iFilter.
- Realiza una serie de procesos como: tokenización, detección de lenguaje, extracción de entidades, stopwords, stemming o lematización, etc.
- Escribe información en la base de datos de Links (B) para formar un Web Graph y poder usarlo en el modelo de ranking. Es decir, utiliza el mismo concepto que los motores de búsqueda de Internet para mejorar los ranking de resultados.
- Además también Genera variaciones fonéticas para la búsqueda de personas.

3. Componente de Procesamiento de Analíticas (Analytics Procesing)

- Analiza los elementos rastreados y cómo los usuarios interaccionan con los resultados de búsqueda (Analytics Reporting Database - C). Por ejemplo, cuando un usuario ve una página ese evento se recoge en el (Event Store) y este componente puede usar esto para analizar comportamientos y mejorar el ranking consecuentemente.
- Podemos crear nuestros propios eventos personalizados
- También utiliza la base de datos de Link (B) para combinar esta información con la información de uso y obtener así mejoras para el algoritmo de ranking.
- Concretamente se realizan los siguiente análisis:
    - Search Analysis
         - Link and Anchor Text Analysis
         - Click Distance
         - Search Clicks
         - Deep Links
         - Social Tags
         - Social Distance
         - Search Reports
   - Usage Analysis
         - Recommendations
         - Usage Counts
         - Activity Ranking

4. Componente de Indexación (Index Component)

- Se encarga de obtener los elementos rastreados y procesados y escribirlos de forma adecuada en los ficheros de índice.
- También recibe las consultas de usuario en un formato compatible y comparable con el formato en que se almacenaron lo elementos. De esta forma es capaz de comparar la consulta del usuario con todos los documentos que tiene almacenados en el índice y devolver el conjunto de documentos (resultados) más adecuado.

5. Componente de Procesamiento de Consulta (Query Processing Component)

- Realiza el procesamiento lingüístico en tiempo de consulta (word breaking, stemming, query spellcheking, expansión de la consulta [thesaurus]).
- Utiliza el modelo de similitud para convertir y adaptar la consulta en un formato adecuado y comparable con los documentos que existen en el índice.
- Optimiza la precisión y relevancia del motor de búsquedas
- Decide cuales de las “Reglas de Consulta” (nuevo término) son aplicables.
- Devuelve a la aplicación cliente los resultados de búsqueda

6. Administración de Búsqueda (Search Administration)

- Responsable del aprovisionamiento y cambios en la topología de servidores de búsqueda
- Responsable de la coordinación de los distintos componentes mencionados anteriormente.
- La Base de datos de Admin Search (D) almacena información acerca de:
    - Topología
    - Crawl Rules
    - Query Rules
    - Managed Properties Mappings
    - Content Sources
    - Crawl Schedules
    - Configuraciones de Analytics
- A nivel informativo, los procesos de búsqueda son:
    - Host Controller: Servicio de Windows que supervisa los procesos NodeRunner.
    - NodeRunner.exe: Proceso que contiene los componentes de búsqueda (uno por cada componente: Crawl, Content Processing, Query, Index, Analytic). Puede haber más de uno por servidor.
    - MSSearch.exe: Windows Service que contiene el componente de Crawl

Using FASTSearch for SharePoint 2010 to index Exchange public folders

 

Indexing of Exchange public folders is one of the less used options in FASTSearch for SharePoint. At least so it seems; I can't find any examples of people setting it up.
In this example I'm using the SharePoint 2010 demo VHDs available here: 2010 Information Worker Demonstration and Evaluation Virtual Machine (SP1). Please note that there are no public folders available on the Exchange Server available in this package, so you have to enable the public folder database and create some public folders by yourself.
First, create the content source:
1. Go into your FAST Content SSA (FASTContent) and create a new content source.
2. Give your content source a name, and choose "Exchange Public Folders".
3. Type the start address; this is the tricky part. To find it, go to your OWA, in my case https://demo2010b/exchange. Click on Public Folders in the left bottom corner. In the new view right click on Public Folders and choose "Open in New Window". The URL you get is your start URL. It is ugly, mine is: https://demo2010b/owa/?ae=Folder&t=IPF.Note&id=PSF.LgAAAAAaRHOQqmYRzZvIAKoAL8RaAwAryKKxImnhRaK28blMCPQ0AAAAAAABAAAB&pspid=_1317728045392_959158536

Second, create a crawl rule to make sure the crawler authenticates correctly.
1. Go into Crawl Rules in your Content SSA.
2. Click "New Crawl Rule"
3. Input the PATH of your OWA, in by case https://demo2010b/*
4. Choose "Include all items in this path"
5. Specify authentication; in my case I used the default content access account.
Then start crawling, enjoy!

How to search Exchange Public Folder using SharePoint Search

 

SharePoint 2010 Search can search multiple location. One of those location/content source is Exchange Public Folders.
This article talks about steps that are required to search public folders in Exchange 2007 SP2 or later (which includes Exchange 2010).
Step1: To get the URL of Public Folder do the following:(assuming that you have already created public folder)

• Open Exchange OWA.
• Click on Public Folders in the Folder list of your mailbox.
• Right Click on the Public Folder you want SharePoint to index and choose open in New Window
• Copy entire URL of the Public Folder in the clipboard.

Step2: Go to Central Administration-> Click on Manage Service Application under Application Management.
Step3: Highlight Search Service Application in the list of Service Application. Then click manage.
Step4: Under Crawling, click on Crawl Rules. Click on New Crawl Rule.

• Paste the Public Folder URL in the Path.
• Choose "Match Case"
• Under Crawl Configuration, choose Include all in this path and then choose "Crawl complex URL (URLs that contain a question mark -?)
• Under Specify Authentication click on Use the default content access account.
• Click OK

Step5: Click on content source under crawling -> New Content Source. Give it a name and then choose Exchange Public Folders. In the start address paste the Public Folder URL.
Step6:Under Crawl Settings choose "Crawl everything that comes under the hostname".
Step7: Set the Crawl schedule according to your convenience. In the content source priority choose normal.
Step8:  Under Start Full Crawl, check it.
Step9: click OK
And its done!!!...
Next time do you a search for content in public folder it will be visible in the search results. (If you have atleast READ permission on the public folder :) )