Add SharePoint 2010 Search Crawl and Managed Properties using PowerShell

The following PowerShell script can be used to add Search Crawl properties and Managed properties.  The properties are added based on vales read from an XML file.  This script comes in handy if you have multiple properties you need to add to multiple environments.

PowerShell Script

1. #script reads from xml file to add crawled properties and managed properties
2. cls
3. #add sharepoint cmdlets
4. if ( (Get-PSSnapin -Name Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue) -eq $null )
5. {    
6.       Add-PsSnapin Microsoft.SharePoint.PowerShell
7. }
8.  
9. #get the XML file
10. [System.Xml.XmlDocument] $XmlDoc = new-object System.Xml.XmlDocument
11. $file = resolve-path(".\searchprop.xml")
12. if (!$file)
13. {
14.         Write-Host "Could not find the configuration file specified. Aborting." -ForegroundColor red    
15.         Break
16. }
17.  
18. write-host "Parsing file: " $file
19. $XmlDoc = [xml](Get-Content $file)
20.  
21. #get the node containing the name of the search service application where you want to add properties
22. $sa = $XmlDoc.SearchProperties.ServiceName
23. $searchapp = Get-SPEnterpriseSearchServiceApplication $sa
24.  
25. #loop through crawled properties to check or add -- don't add if it already exists
26. $CrawledPropNodeList = $XmlDoc.SearchProperties.CrawledProperties
27.  
28. foreach ($CrawledPropNode in $CrawledPropNodeList.CrawledProperty)
29. {
30.     $SPCrawlProp = $CrawledPropNode.Name
31.     $SPCrawlPropType = $CrawledPropNode.Type
32.     
33.     #Create Crawled Property if it doesn't exist
34.     if (!(Get-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Name $SPCrawlProp -ea "silentlycontinue"))
35.     {
36.         switch ($SPCrawlPropType)
37.         {
38.         "Text" {$crawlprop = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Category SharePoint -VariantType 31 -Name $SPCrawlProp -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346"}
39.         "Integer" {$crawlprop = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Category SharePoint -VariantType 20 -Name $SPCrawlProp -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346"}  
40.         "Decimal" {$crawlprop = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Category SharePoint -VariantType 5 -Name $SPCrawlProp -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346"}  
41.         "DateTime" {$crawlprop = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Category SharePoint -VariantType 64 -Name $SPCrawlProp -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346"}
42.         "YesNo" {$crawlprop = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Category SharePoint -VariantType 11 -Name $SPCrawlProp -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346"}
43.         default {$crawlprop = New-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Category SharePoint -VariantType 31 -Name $SPCrawlProp -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346"}
44.         }
45.     }
46. }
47.  
48. #now that the crawled properties exist, loop through managed properties and add
49. $PropertyNodeList = $XmlDoc.SearchProperties.ManagedProperties
50.  
51. foreach ($PropertyNode in $PropertyNodeList.ManagedProperty)
52. {
53.     $SharePointProp = $PropertyNode.Name
54.     $SharePointPropType = $PropertyNode.Type
55.     $SharePointPropMapList = $PropertyNode.Map
56.     #add managed property
57.     #remove it if it already exists
58.     if ($mp = Get-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $searchapp -Identity $SharePointProp -ea "silentlycontinue")
59.     {
60.          #$mp | Remove-SPEnterpriseSearchMetadataManagedProperty -Confirm
61.         $mp.DeleteAllMappings()
62.         $mp.Delete()
63.         $searchapp.Update()
64.     }
65.     New-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $searchapp -Name $SharePointProp -Type $SharePointPropType
66.     $mp = Get-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $searchapp -Identity $SharePointProp
67.     #add multiple crawled property mappings
68.     foreach ($SharePointPropMap in $SharePointPropMapList)
69.     {
70.         $SPMapCat = $SharePointPropMap.Category
71.         $SPMapName = $SharePointPropMap.InnerText
72.         $cat = Get-SPEnterpriseSearchMetadataCategory –SearchApplication $searchapp –Identity $SPMapCat
73.         $prop = Get-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchapp -Category $cat -Name $SPMapName
74.         New-SPEnterpriseSearchMetadataMapping -SearchApplication $searchapp -CrawledProperty $prop -ManagedProperty $mp
75.     }
76. }

Here’s a sample of the XML file where the properties to be added can be read from:

searchprop.xml Sample File

1. <?xml version="1.0" encoding="utf-8"?>
2. <SearchProperties>
3.     <ServiceName>Search Service Application</ServiceName>
4.   <CrawledProperties>
5.     <CrawledProperty Type="Text" Name="ows_SiteDescription" />
6.     <CrawledProperty Type="Decimal" Name="ows_MemberCount" />
7.     <CrawledProperty Type="Text" Name="ows_SiteURL" />
8.     <CrawledProperty Type="Decimal" Name="ows_SiteHits" />
9.     <CrawledProperty Type="YesNo" Name="ows_ProjSensitive" />
10.     <CrawledProperty Type="DateTime" Name="ows_ProjStartDate" />
11.     <CrawledProperty Type="Text" Name="ows_ProjectName" />
12.   </CrawledProperties>
13.     <ManagedProperties>
14.         <ManagedProperty Type="1" Name="XYZSiteDesc">
15.             <Map Category="SharePoint">ows_SiteDescription</Map>
16.         </ManagedProperty>
17.         <ManagedProperty Type="3" Name="XYZMembers">
18.             <Map Category="SharePoint">ows_MemberCount</Map>
19.         </ManagedProperty>
20.         <ManagedProperty Type="1" Name="XYZSiteUrl">
21.             <Map Category="SharePoint">ows_SiteURL</Map>
22.         </ManagedProperty>
23.         <ManagedProperty Type="3" Name="XYZSiteHits">
24.             <Map Category="SharePoint">ows_SiteHits</Map>
25.         </ManagedProperty>
26.     <ManagedProperty Type="5" Name="XYZProjectSecurity">
27.       <Map Category="SharePoint">ows_ProjSensitive</Map>
28.     </ManagedProperty>
29.     <ManagedProperty Type="4" Name="XYZProjectStartDate">
30.       <Map Category="SharePoint">ows_ProjStartDate</Map>
31.     </ManagedProperty>
32.     <ManagedProperty Type="1" Name="XYZProjectName">
33.       <Map Category="Basic">urn:schemas.microsoft.com:fulltextqueryinfo:displaytitle</Map>
34.       <Map Category="SharePoint">ows_Title</Map>
35.     </ManagedProperty>
36.   </ManagedProperties>
37. </SearchProperties>

Note:  The PowerShell script and XML file should be placed in same folder prior to running.

References:

Technet link for cmdlet New-SPEnterpriseSearchMetadataCrawledProperty to add a new crawl property.

Technet link for cmdlet New-SPEnterpriseSearchMetadataManagedProperty to add new managed property.

Technet link for cmdlet New-SPEnterpriseSearchMetadataMapping to add new mapping to a managed property.

Managed Property type key values are as follows:

• Text = 1
• Integer = 2
• Decimal = 3
• DateTime = 4
• YesNo = 5
• Binary = 6

Remove and Add Back Multiple SharePoint Content Databases with PowerShell

The TechNet documentation for deployment of software updates for SharePoint Server 2007 suggests, among other things, to detach your content databases in order to reduce downtime. This is easy enough to do through Central Admin when you have a few databases and a small number of web applications, but how about in a large farm with many web applications and dozens of content databases with multiple SQL Server Instances? PowerShell to the rescue!

First: Remove/Detach the Databases – Create a CSV file with value headings webapp and url where webapp is friendly name and url is the associated url for the web application. CSV example below:

webapp,url

webApp1,http://webapp1.xyz.com

webApp2,http://webapp2.xyz.com

Use below PowerShell script to loop through all of your web applications using above CSV file, enumerate the content databases via STSADM, and throw the content database name and SQL Server Server/Instance name to a new CSV file named by webapp value. An internal loop reads from the new CSV file and detaches the databases (I commented line out below – uncomment when ready to run).

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

#loop through a csv file that contains webapp and url, run enumcontentdbs, export results to csv  
Import-Csv .\webapps.csv | ForEach {   
    $strWA = $_.webapp  

    $strURL = $_.url 

    $rawdata=stsadm -o enumcontentdbs -url $strURL 

    $sitesxml=[XML]$rawdata 

    $sitesxml.databases.contentdatabase | Select-Object server,name | export-csv .\$strWA.csv -noType 

    #loop through contentDBs from csv created in above line and delete 

    Import-Csv .\$strWA.csv | ForEach { 

        $strServer = $_.Server 

        $strName = $_.Name 

        write-output WebApp: $strURL ContentDB: $strName 

        #WARNING BELOW COMMAND DANGEROUS -- UNCOMMENT WHEN READY -- WILL REMOVE EVERY CONTENT DB ON FARM 

        #$delDB = stsadm.exe -o deletecontentdb -url $strURL -databasename $strName 

    } 

}

Second: Add the Databases back – After you’ve run all of your updates, you need to add your content databases back. Use below PowerShell script to loop through all of your web applications and read in the values from the CSV files created by running previous script to add the content databases back.

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") 

#loop through webapps from original csv -- same as remove script
Import-Csv .\webapps.csv | ForEach {

    $strWA = $_.webapp 

    $strURL = $_.url 

    $rawdata=stsadm -o enumcontentdbs -url $strURL 

    #loop through contentDBs and add based on csv created in the webAppDbs_Remove.ps1

    Import-Csv .\$strWA.csv | ForEach {

        $strServer = $_.Server 

        $strName = $_.Name 

        #write-output WebApp: $strURL ContentDB: $strName

        $AddDB = stsadm.exe -o addcontentdb -url $strURL -databasename $strName -databaseserver $strServer -sitewarning 450 -sitemax 500
    } 

}

Add the scipts in appropriately named powershell ps1 files (e.g., webAppDBs_Remove.ps1 and webAppDBs_Add.ps1) on an Application or Web Front End Server where Powershell is installed and call from a .bat file with commands similar to below:

powershell.exe Set-ExecutionPolicy RemoteSigned

REM command to run script from the ps1 file to remove

powershell.exe -noexit .\webAppDBs_Remove.ps1

REM command to run script from the ps1 file to add back

REM powershell.exe -noexit .\webAppDBs_Add.ps1

Update SharePoint web.config with Powershell

Had a need to update 48 web.config files in a SharePoint farm. Purpose was to add Novell E-Directory as an authentication provider for SharePoint Farm in our DMZ consisting of 4 web applications (8 IIS websites by virtue of extending) load balanced across 6 Web Front Ends (more on that in a future post).

To add the provider, there are several web.config additions...and I'm sick of writing complex checklists for our operations folks to carry out the changes. I was also a little worried about them missing some of the additions. I prefer to have them perform one step -- double-click "webconfigupdate.bat" -- and be done rather than manually having them open, edit, and save 48 web.config files.

Powershell and the SPWebConfigModification Class to the rescue.

• Download and install Powershell 1.0 here (you only have to install on one server in your farm where SharePoint is installed -- wherever you have Central Administration site running is fine).
  
• MSDN info about SPWebConfigModification is here.
  

Credit needs to go to following -- I was able to merge the ideas from the code in the first two posts to get my final powershell script:

• This is post by Raymond Mitchell that got me started. It's great sample Powershell script for single web app. Although, I needed to loop through every web app in my farm.
  
• This is the post by Gary Lapointe where I found how to use Powershell to loop through web apps on a farm – it's not the first time Gary's blog helped me understand some administration scripting.
  
• Mark Wagner has some nice guidance on this topic. Helped me understand errors I received.
  
• Reza Alirezaei has a great post on SPWebConfigModification's Top 6 Issues which helped me confirm much of the oddness that resulted when testing.
  
• Gotta thank "The Groom," a team member who got the SPWebConfigModification class working in a Solution he deployed. I tried the SPWebConfigModification class once before but abandoned it because of issues I was seeing based on Reza's post. The Groom's success inspired to give it another try.
  

Before starting, SPWebConfigModification Class requires XPath values for its "Path" and "Name" Properties. The Path and Name values are how the Class finds the location in the web.config to add, update, and remove values.

The BIT-101 XPath Query Tool is a nice, free, on-line tool for finding correct XPath to enter. I used the tool by simply copying my web.config – with new elements and attributes I wanted in my final web.config -- and pasting it into this java based tool. I than queried for the new elements and attributes to determine the XPath I would use in my script.

There are some nice examples here to help you understand XPath if you're not familiar.

Here's the code I threw into a Powershell .ps1 file I named webconfigchange.ps1:

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
$farm = [Microsoft.SharePoint.Administration.SPFarm]::Local
$websvcs = $farm.Services  where -FilterScript {$_.GetType() -eq [Microsoft.SharePoint.Administration.SPWebService]}
#use powershell method for suppressing errors for reason on next line -- updates will run successfully even though errors are thrown -- comment out erroractionpreference variable when debugging
#If more than one call to the ApplyWebConfigModifications() method is made in close succession you will get the following error due to timer job:
#A web configuration modification operation is already running.
$erroractionpreference = "SilentlyContinue"
 
#loop through each portal web app
foreach ($websvc in $websvcs)
{
foreach ($webapp in $websvc.WebApplications)
    {
    #update web.config for all web apps in farm excluding mysites and ssp
    if ($webapp.name -like "*my*") {write-output "skipping mysites..."}
    elseif ($webapp.name -like "*ssp*") {write-output "skipping SSP..."}
    else
        {
        #add peoplepicker node for IDM – IDM is name we will use for Authentication Provider for Novell Identity Management
        #here's where you'll need to understand XPath – this variable captures the xpath query – later in script, I'll optionally use Name and Path instead of Argumentlist
        $ppMod = New-Object -TypeName "Microsoft.SharePoint.Administration.SPWebConfigModification" -ArgumentList "add[@key='IDM']", "configuration/SharePoint/PeoplePickerWildcards"
        $ppMod.Sequence = 0
        $ppMod.Owner = "SimpleSampleUniqueOwnerValue"
        $ppMod.Type = "EnsureChildNode"
        #this is actual value which will be added or modified
        $ppMod.Value = "<add key='IDM' value='*' />"
        $webapp.WebConfigModifications.Add($ppMod)
 
        #add membership node
        $memberMod = New-Object -TypeName "Microsoft.SharePoint.Administration.SPWebConfigModification" -ArgumentList "membership", "configuration/system.web"
        $memberMod.Sequence = 0
        $memberMod.Owner = "SimpleSampleUniqueOwnerValue"
        $memberMod.Type = "EnsureChildNode"
        $memberMod.Value = "<membership defaultProvider='IDM'></membership>"
        $webapp.WebConfigModifications.Add($memberMod)
  
        #add providers node
        $providerMod = New-Object -TypeName "Microsoft.SharePoint.Administration.SPWebConfigModification" -ArgumentList "providers", "configuration/system.web/membership"
        $providerMod.Sequence = 0
        $providerMod.Owner = "SimpleSampleUniqueOwnerValue"
        $providerMod.Type = "EnsureChildNode"
        $providerMod.Value = "<providers></providers>"
        $webapp.WebConfigModifications.Add($providerMod)
 
        #add IDM LDAP attributes – here, we're using Path and Name properties instead of Argumentlist
        $ldapMod = New-Object Microsoft.SharePoint.Administration.SPWebConfigModification
        $ldapMod.Path = "configuration/system.web/membership/providers"
        $ldapMod.Name = "add[@name='IDM'][@type='Microsoft.Office.Server.Security.LDAPMembershipProvider, Microsoft.Office.Server, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C'][@server='<your Ldap server>'][@port='389'][@useSSL='false'][@useDNAttribute='false'][@userDNAttribute='cn'][@userNameAttribute='mail'][@userContainer='<your Ldap path>'][@userObjectClass='person'][@userFilter='(ObjectClass=*)'][@scope='Subtree'][@otherRequiredUserAttributes='sn,givenname,cn']"
        $ldapMod.Sequence = 0
        $ldapMod.Owner = "SimpleSampleUniqueOwnerValue"
        $ldapMod.Type = "EnsureChildNode"
        $ldapMod.Value = "<add name='IDM' type='Microsoft.Office.Server.Security.LDAPMembershipProvider, Microsoft.Office.Server, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C' server='<your Ldap server>' port='389' useSSL='false' useDNAttribute='false' userDNAttribute='cn' userNameAttribute='mail' userContainer='<your Ldap path>' userObjectClass='person' userFilter='(ObjectClass=*)' scope='Subtree' otherRequiredUserAttributes='sn,givenname,cn' />"
        $webapp.WebConfigModifications.Add($ldapMod)
 
        #save changes and apply to farm
        $method = [Microsoft.Sharepoint.Administration.SPServiceCollection].GetMethod("GetValue", [Type]::EmptyTypes)
        $closedMethod = $method.MakeGenericMethod([Microsoft.Sharepoint.Administration.SPWebService])
        $services = $webapp.Farm.Services
        $service = $closedMethod.Invoke($services, [Type]::EmptyTypes)
        $service.ApplyWebConfigModifications()
        $webapp.Update()
        }
    }
}

I call the above script to run from a .bat file with below commands:

powershell.exe Set-ExecutionPolicy RemoteSigned
REM command to run script from the ps1 file
powershell.exe -noexit .\webconfigchange.ps1

So there it is. With double-click of batch file, 48 web.config files are modified. If I ever want to remove the values, I can run same ps1 but replace every "WebConfigModifications.Add" entry with "WebConfigModifications.Remove." Another thing I like about SPWebConfigModification Class is that all the changes are stored in the SharePoint configuration database so if I ever add new front ends, the web.config values are automatically populated.