Tuesday, December 27, 2011

Drive with FAST search for SharePoint 2010 installation is running out of space

On one of the projects I was seeing recurring behavior where “D:\FAST Search\data\ftStorage\sequences” folder quickly fills up with files like storage_6a.data , storage_cd.meta thus causing the drive to run out of space. That location is used to store index (“fixml”) information used for fault tolerance (ft) in a backup indexing row. We have deleted these files at some point since there was an environment health issue as well, hoping that fixing the environment issue was to blame for causing this folder to fill up, but it came back again and quickly filled up the drive. In searching for the cause of the problem here is what was found.

1. It’s likely this is happening if one or more servers are configured as a “secondary” indexing row.  See this example from a FS4SP deployment.xml file:

<searchcluster>

                <row id="0" index="primary" search="true" />

                <row id="1" index="secondary" search="true" />

</searchcluster>

The options are to disable the secondary/backup indexer(s), or add more disk space to the backup row.

If the data was deleted in this folder, it’s likely the backup index data is out of sync, and will need to be resynchronized with this procedure:

“Synchronize the primary and the backup indexer servers (FAST Search Server 2010 for SharePoint)” - http://technet.microsoft.com/en-us/library/gg482028.aspx

2. In our case the box that had this issue was not configured as a secondary row, but was a primary row.

The deployment.xml has not used row 0 as the primary row (which is required, ref http://technet.microsoft.com/en-us/library/ff354931.aspx#element_searchcluster)

This caused the ftstorage files to neither be placed on the intended server, nor rotated and thus eating up the disk space.

Here is an example of the deployment

<host name="FAST01 ">

<admin/>

<crawler role="single"/>

<webanalyzer lookup-db="true" link-processing="true" max-targets="2" server="true"/> <document-processor processes="4"/> <content-distributor id="0"/> <indexing-dispatcher/>

<query/> <searchengine column="0" row="0"/> </host>

<host name="FAST02 ">

<document-processor processes="4"/>

<content-distributor id="1"/> <indexing-dispatcher/>

<query/>

<searchengine column="0" row="1"/>

</host>

<searchcluster>

<row id="1" search="true" index="primary"/> <row id="0" search="true" index="secondary"/>

</searchcluster>

Once the IDs were reconfigured and then I ran Set-FASTSearchConfiguration on the admin box as well as the non-admin. The issue went away Smile

Enjoy

Friday, December 23, 2011

NOINDEX Handling by FAST and SharePoint Search

Just recently I had to go through troubleshooting of content being indexed by FAST and/or SharePoint Search and wanted to share experience and some collected wisdom.

Here is the case, very common: the client has branded master page that includes menu navigation items, these items were being indexed. Where in case of searching for “Vacation form” would bring every page that is using this master page with menu items, simply because link to vacation form was included into the navigation menu on this master page. After the client included ”noindex”, which works for FAST as well as for SharePoint it seemed to work. Few weeks later when I came onsite, I noticed that the Search SSAs for SharePoint as well as for FAST had their content sources misconfigured and without even knowing the client was using just SharePoint search for everything. At this point I’ve reconfigured the content sources so only “people” search is being server by SharePoint search and the rest of the content is searched by FAST. This is when we have noticed the same problem where menu items were being indexed again without respecting the NOINDEX tag. Once we purged the index and reindexed everything again, it all worked.

Here are the tags and the explanation of how they work:

1) <meta name="robots" content="noindex" /> : Supported using both crawlers, although differently. FAST crawler will drop these items. SP crawler will not drop the items, they will be dropped by the FAST pipeline. But the result is the same for both.

2) <noindex> This text will not be indexed </noindex> : Not supported regardless of crawler.

3) <div class =”noindex”> This text will not be indexed </div> : Supported. This is transparent to both crawlers, will be filtered in the FAST pipeline.

4) <span class =”noindex”> : Supported by FAST Search back-end but not SharePoint search.

5) SP crawler fix to ensure that the <meta name="xxx" content="noindex" /> was passed to the FAST pipeline (KB 2276336)

http://support.microsoft.com/kb/2276336

Part of the August 2010 CU.

Hope this helps to others.

Enjoy Holidays!!!!

Smile

Monday, October 10, 2011

FAST Search for SharePoint 2010 administration screenshots (Central Administration)

Many times I get a question as to how granular can you make the administration of the FAST, what administrative options each level provides and what does it look like.

In the previous post I shared screenshots of administrative interfaces that are available to site collection admins. In this post I wanted to share the screenshots of Administration UI for FAST from the Central administration.

Search Administrators have the ability to control the following settings:

Here are some screenshots with explanations.

Central Administration – Content source configuration

Add Content Source

Central Administration – Configure authoritative pages

Configure Authoritative Pages (Relevance Ranking)

Central Administration – Content Source overview

Content Sources Overview

Central Administration – Crawl log

Crawl Log

Central Administration – Crawled Properties (Metadata)

Crawled Properties (Metadata)

Central Administration – FAST Content SSA overview

FASTContent Overview

Central Administration – FAST Query SSA overview

FASTQuery Overview

Central Administration – Managed Properties (metadata fields)

Managed Properties (Fields)

Central Administration – New Crawl rule

New Crawl Rule

Central Administration – URL Blocking

URL Blocking

Enjoy Smile

Wednesday, September 7, 2011

FAST Search for SharePoint 2010 administration screenshots (Site settings)

Many times I get a question as to how granular can you make the administration of the FAST, what administrative options each level provides and what does it look like.

In this post I wanted to share the screenshots of Administration UI for FAST from the site collection admin level as well as Central administration.

Site collection administrators have ability to provide administration for:

  1. Best Bets
  2. Visual Best Best
  3. Content promotion and demotion
  4. User context

You do not need to make your business folks (that would be more suitable to customize the search experience) Search Admins and give them access to Central Administration site where they can become quite dangerous. you can get access to the following administration areas through the site settings page on the site collection level.

Here are some screenshots with explanations.

Site collection – Site Settings

Best Bets configuration

Add Best Bet

Keyword, Phrases and Synonyms configuration

Add FAST Search Keyword and Configure Synonym

Add site demotion

Add Site Demotion

Add site promotion (boosting content from a site)

Add Site Promotion (Boosting)

Site Promotion (boosting)

Add user context

Add User Context

Manage Use Context

User Context

Add visual Best Bet

Add Visual Best Bet

In the next post I’ll put the screenshot of administrative interfaces that are available for the Search Admins from the Central Administration site

Enjoy Smile

Wednesday, August 3, 2011

SPC 2011 : Making the Move to SharePoint 2010 - Search First

I’m speaking at the SPC in Anaheim, California in October. SPC had be always to me THE SharePoint Conference to attend. If you still did not register you might be missing the most important SharePoint event of the year.

If you are there, check out my session that I’ll be doing along side with Mark Stone (Microsoft) and Harry Jones (Arcovis) and learn one upgrade strategy for 2010 that no one told you about.

Here is the session abstract.

Most of us have heard about the enhancements and power offered by SharePoint 2010 Enterprise Search, but many do not realize they can take advantage of it before upgrading their 2007/MOSS environment. In this session we will cover how to leverage Search for SharePoint 2010 and FAST Search for SharePoint today as part of your organizations roadmap to 2010. We'll demonstrate the integration of SharePoint 2007 and SharePoint Search 2010 from the end-user perspective and discuss the many benefits “Search First” offers to an organization and their users.

http://www.mssharepointconference.com/pages/sessiondetailsShort.aspx?sessionguid=90a461cd-d717-45be-b09b-c82a1f9fefdc

Why Social Networking on SharePoint 2010?

I recently wrote an article on social networking on SharePoint platform and wanted to share it with you.

The SharePoint 2010 platform offers a wide variety of core features including document management, records management, BI, search, workflow and social networking to name a few. Microsoft and its partners have done a great job of showcasing the value of these features and have shared strategies to help organizations take advantage of them. However, of these core features, it is my observation that the Social Networking aspects of SharePoint 2010 are underutilized by customers and therefore the full potential of their implementations are not realized.

The most common objection I have seen to the notion of social networking within the enterprise has to do with a misconception that social networking platforms are not productive in the work environment. I often see is a misguided comparison made to Facebook and other external social networking sites which management views as time-wasters. Other barriers to adoption include uncertainties about allowing uncensored comments by users, a difficulty in justifying the investment and the struggle involved with getting buy-in from the business.

Read more on MVP blog http://blogs.msdn.com/b/mvpawardprogram/archive/2011/07/25/mvps-for-office-365-why-social-networking-on-sharepoint-2010.aspx

Monday, June 27, 2011

Social Sites Demo on next Shoptalk: June 27th, 2011

Social Sites for SharePoint 2010
presented by: Rich Blank from News Gator

By integrating Social Sites directly into your SharePoint 2010 collaboration deployment, you can focus on delivering value to your users and unifying your technology infrastructure while fully leveraging your SharePoint social network - instead of building custom applications or learning and supporting additional technologies. Increase adoption among stakeholders, streamline communication, improve employee recruiting and retention, and boost overall productivity with Social Sites 2010.
Speaker Bio
For over 15 years, Rich Blank has worked with hundreds of organizations evangelizing, architecting and delivering social computing, collaboration & content management solutions. His passion is helping organizations change the way people connect, share information, and manage knowledge and his expertise includes information & systems architecture, end user adoption, governance, business process improvements, and technology strategy. Rich is a featured MS SharePoint expert and regular blogger on AIIM.org (Association of Information & Image Management) writing about collaboration, ECM, and social business. He's also has been invited as a guest speaker at the 2008 IT & Business Alignment Conference, 2010 SharePoint Summit, and regional SharePoint User Groups. He currently works as a Solutions Engineer for Newsgator and has held past senior technology and management consulting roles at EMC and IBM. Rich obtained his MBA from Babson College and BBA from the University of Michigan, and is a certified PMP and Six Sigma Green Belt. You can read some more of his insights on his personal blog: www.pmpinsights.com, follow him on Twitter: @pmpinsights, or connect with him on LinkedIn: www.richblank.com


SharePoint ShopTalk is a live, dynamic, discussion held weekly amongst SharePoint Professionals.  It is an excellent source for users who need to discuss challenges or are looking for answers to pressing contemporary SharePoint questions. All are welcome to take advantage of this forum that features leading experts in the field.
You may register today and join the SharePoint ShopTalk Group on LinkedIn to submit questions prior to the event. You may also tweet your questions to @spshoptalk as well as ask questions live! We encourage participants to send questions in advance so our panel of experts may answer at their highest capacity.
For this event, you may also tweet your questions to pmpinsights & NewsGator.

Enjoy Smile

Monday, June 20, 2011

Error installing FAST Search for SharePoint 2010

I’ve ran into a lot of installation errors in my experience, but this one definitely deserves a separate blog post. During the installation I’ve got the following error

06/13/2011 11:25:23 AM Warning Utility.Execute - "D:\FASTSearch\bin\ResourceStoreInstaller.exe" StdError - resourcestoreinstaller.exe - C:\Windows\system32\inetsrv\appcmd.exe SET config "ResourceStore" /section:requestFiltering /verbs.allowUnlisted:false /commit:apphost - ERROR ( message:Unknown attribute "verbs.allowUnlisted". Replace with -? for help. )

06/13/2011 11:25:23 AM Warning Utility.Execute - Return code for binary "D:\FASTSearch\bin\ResourceStoreInstaller.exe" is not 0. This may indicate that binary didn't execute successfully

And the FAST failed to install.

Let me step back a little bit, the installation was fairly simple, just 2 servers. And knowing the FAST hungry for resources reputation we usually recommend to beef up the servers as much as possible. The 2 servers were actually physical machines very powerful with about 16 CPU cores registered. That was the problem!

Christian Holslin shared his experience here, otherwise I’d be chasing the problem for hours. http://www.thesharepointblog.net/Lists/Posts/Post.aspx?List=815f255a%2Dd0ef%2D4258%2Dbe2a%2D28487dc9975c&ID=48

Too many CPU cores, or logical CPUs can produce the failure for the installation as the CPUs compete for access to the web.config file.

TO resolve this problem do the following:

Before you attempt to run the configuration wizard, from the command line, run Mdconfig command:

1. Go to advanced tab

2. Under advanced go to “boot”

3. Select 4 processors

4. Restart the machine. Repeat this process for all servers in the FAST farm.

Once the installation process is done, revert your changes for the number of registered CPUs back to the original amount of registered Cores following the steps above.

You will be all set

Enjoy Smile

Friday, June 10, 2011

SharePoint 2010 Enterprise Search isn’t just for SharePoint 2010–presentation is available

Most of us have heard about the enhancements and power offered by SharePoint 2010 Enterprise Search, but many do not realize they can take advantage of it without upgrading their 2007/MOSS environment. In this session we will cover how to leverage Search for SharePoint 2010 and FAST Search for SharePoint today, demonstrate the seamless integration of SharePoint 2007 and SharePoint Search 2010 from the end-user perspective and discuss the many benefits “Search First” offers to an organization and their users.
Here is a link to the presentation
http://www.slideshare.net/natalyvo/share-pointesug-searchfirst61011

Enjoy Smile

Thursday, May 26, 2011

Opportunity for Senior SharePoint Consultants

Arcovis is seeking Senior SharePoint Consultants to join its team of SharePoint and Enterprise Search experts. This position offers the opportunity to work with leading minds in the industry, exposure and access to Microsoft account and technical teams, C-level executives at fortune 100 companies, as well as broad industry exposure. This is an excellent opportunity to learn new technologies, latest trends in the industry, and develop influential relationships. For more information see the full job description.

Technorati Tags:

Sunday, May 15, 2011

SharePoint FEST, Denver–May 19th session: Integration of the SharePoint Search 2010 and FAST Search for SharePoint 2010 with SharePoint 2007 portal

I’m speaking at the SharePoint Fest in Denver on May 19th. I’ll be presenting on one of my favorite topics, here is the session topic and the abstract:

Integration of the SharePoint Search 2010 and FAST Search for SharePoint 2010 with SharePoint 2007 portal

With the release of the SharePoint 2010, Microsoft had release the true enterprise search solution, FAST Search for SharePoint 2010 (FS4SP). But what about companies who have invested heavily in 2007 implementations and while planning to upgrade in the future want to take advantage of FAST Search for SharePoint 2010 now. This session covers the integration of SharePoint 2007 and FAST search for SharePoint 2010 and benefits from the business and end-user perspective that Search 2010 brings into the 2007 portal solution.

The session is going to be informative to everyone and not just IT Pros and Devs, it will also include demos.

Come and say Hi. I’ll see you there

Technorati Tags:

Tuesday, April 26, 2011

FAST Search: the infamous “Failed to initialize session with document engine: Unable to resolve Content distributor”

In the numerous FAST configurations the error above is the most common and one that I wanted to share my experiences with. There is a number of issues that might be causing this error and renders FAST Search unable to crawl. This issue can be from the initial configuration of FAST, but can also be introduced later on if you modify the deployment.XML and reconfigure FAST.

Here is the list of the most common ways of resolution and is in the order of :

1. Secure the connector by running

.\SecureFASTSearchConnector.ps1 -certPath "path of the certificate\certificatename.pfx" -ssaName "name of your content SSA" -username "domain\username"

replacing the necessary parameters with the values for your environment. Here are the detailed instructions http://technet.microsoft.com/en-us/library/ff381261.aspx

2. Check if the domain and user name is reflecting the details of the user running the SharePoint Server Search 14 (OSearch14) service. The user account running Server Search 14 service should be the same as the user name used to secure the connector.

3. Make sure that the content distributor URL is the same as in the Install_Info file located within your FASTSearch folder or the same as in the fastsearch\etc\contentdistributor.cfg  file – sometime that might be the issue is the value in the install_info file is not the same as the cfg file, use the URL from the CFG file.

Hope it helps.

Enjoy Smile

Monday, April 11, 2011

New York Enterprise Search User Group celebrates its Two Year Anniversary

The New York Enterprise Search User Group celebrates its Two Year Anniversary by welcoming back the speaker who started it all, Nathan Treloar!
We hope that you will join the festivities, bring a friend as we end the year on high note and a look into the future of Enterprise Search! 

DECEMBER MEETING - Wednesday, December 14th 5:30PM

REGISTER TODAY

SUBJECT
Video Search and the Rise of Time-coded Metadata
Searching video certainly isn’t a new thing. Youtube has been with us for years now, after all. But when it comes to emerging trends in video search, both on the Web and in the enterprise, there are signs that we are entering a new age. The time-coded nature of video and the ability (or lack thereof) to search and navigate *inside* video content represent a new frontier for search that is only just now coming to light.
In this interactive, demo-rich presentation, Nate Treloar, VP of Strategic Markets for RAMP, will walk through the latest trends in video search and show how the rise of time-coded metadata is changing how people discover and engage with online video.
ABOUT THE SPEAKER
TreloarNathan Treloar, is VP of Technology for Strategic Alliances at RAMP where he is responsible for expanding RAMP’s industry leading Content Optimization solutions into new markets. Prior to RAMP, Nate was the Principal Search Technology Evangelist in the Microsoft Enterprise Search Group and held responsibility for the group’s technology innovation and evangelism programs.
Nate came to Microsoft through the acquisition of FAST, where he was Senior Vice President of Technology and a member of the leadership team that took FAST from an entry level player to a leader in the enterprise search market. In a career spanning more than two decades, Nate has held key executive positions for search, text mining, publishing, and knowledge management software and solutions. He has advised hundreds of world’s largest companies and government agencies on applications of search and search related technologies.
ADDRESS
1290 Avenue of the Americas, Sixth Floor NY, NY 10104

Monday, March 21, 2011

SharePoint ShopTalk Announces New Video Blog! and new panelists!!

Join us for the next SharePoint ShopTalk: Thursday, March 24th 12:30PM - 1:30PM EST.

You may register today and join the SharePoint ShopTalk Group on LinkedIn to submit questions prior to the event. You may also tweet your questions to @spshoptalk as well ask questions live! We encourage participants to send questions in advance so our panel of experts may answer at their highest capacity.
Sign up for this month's Panelist Spotlight - "Tips & Tricks" 3.31.11:
Integration of SharePoint Search 2010 and FAST Search for SharePoint 2010 with SharePoint 2007 Portal

SharePoint ShopTalk Announces New Video Blog!
View last week's upload: .NET Developer Learning SharePoint
New Videos are posted every Thursday!

We look forward to hearing from you as well as the continued sharing of SharePoint knowledge. 
Audio Call-in Number: (760) 569-0111

Audio Passcode: 724468#

Click here to register for the next SharePoint ShopTalk

FAST Search for Internet Sites (FSIS) Deep Dive–April 13, 2011

I’m happy to announce the Enterprise Search User Group meeting for April. Save the date

This session will provide a technical deep dive of FAST Search for Internet Sites focused on the main components: Content Transformation Services (CTS), Interaction Management Services (IMS) and the Enterprise Search Platform (ESP).  Learn what the main components are and how they work together to give you one of the most powerful search platforms on the market.  We will build an end-to-end experience with a focus on explaining how the different components work and interact. 

SPEAKER
Runar Olsen, is a Senior Architect with Microsoft's Enterprise Search Practice.  At this time with Microsoft he has been involved with large scale and complex internet site deployments, and recently focusing on the new Enterprise Search 2010 product lineup. He holds an M.S. in Computer Science from the Norwegian University of Tromsø (UITØ). 
In this talk Runar Olsen will provide a technical deep dive of FAST Search for Internet Sites focused on the main components: Content Transformation Services (CTS), Interaction Management Services (IMS), and the Enterprise Search Platform (ESP).  Learn what the main components are and how they work together to give you one one of the most powerful search platforms on the market. We will build an end-to-end  experience with a focus on explaining how the different components work and interact.

MEETING AGENDA
5:30 PM - Networking (Food and Drinks)

6:00 PM - News and Highlights

6:10 PM - A word from our sponsor

6:30 PM - Speaker Presentation

7:25 PM - Wrap-Up and Give-Aways

7:30 PM - After Meeting Social

ADDRESS
1290 Avenue of the Americas, Sixth Floor NY, NY 10104

To attend this meeting, REGISTER TODAY!

Security Reminder
Register at least 2 days in advance to secure a building pass, otherwise one will be created at building security which will stall your entry.

Monday, February 7, 2011

Behind the Scenes of the Contoso/ Consulting Search-Driven Application Demo

This presentation promises to be one of the best “behind the scenes” on developing search-driven applications.

Do not forget to register for our Enterprise Search User Group meeting, held every second Wednesday of the month at:

1290 Avenue of the Americas, Sixth Floor NY, NY 10104

FEBRUARY MEETING - Wednesday, February 9th, 2011
SUBJECT
Behind the Scenes of the Contoso Consulting Search-Driven Application Demo
SPEAKERS
Richard Frenkel, CTO Ceonex has 30 years experience in the IT field, initially building systems for mainframes and later for the client/server environment....
Theodore Agranat, User Interface Architect and Design Manager is an accomplished manager of design and user interface architect.  He has built the design side of Ceonex from the beginning....
Dimitri Ayrapetov, Senior SharePoint Consultant has seven years of experience in IT, with the past four years primarily working with SharePoint technologies.... 
Read more about our speakers here


This session will provide a technical drilldown of how the Contoso Consulting Search-Driven Application demo was built. The Search Driven Demo builds on the standard FAST Search for SharePoint web parts to create a "Search Driven Application": in this case a company Knowledge Center. The application includes "visual refiners" that present the information in standard text based refiners visually and allow user interaction, a visual "work area" to which specific items from the result set can be saved, and a power point previewer that allows PowerPoint presentations to be viewed and slides selected within the search results, among other customizations. The presenters will discuss the application architecture and custom development efforts required to build this custom Search-Driven Application.
Included in the drilldown will be:
• Brief run through of the demo to show the features
• Breakdown of OOB functionality, extended configuration, and custom developed controls
• Walk through of the custom graphical refiner controls (architecture and code)
• Customization of rendering components
• Implementation of “My Workspace”
• If time and interest allow we may also discuss the use of PowerShell to fully script the configuration of the demo application on SharePoint Enterprise, and how we customized the OOB visual styling.
As part of the discussion, the presenters will discuss pros/cons to the various approaches and challenges faced during implementation. The presentation will include technical drill down, but could be of interest to anyone interested in what is possible in customizing search or building search applications. 
Register Today

Saturday, January 8, 2011

SharePoint Search 101:How does an indexer work? What is Lemmatization? What is Tokenization?

What happens when you index content?

Before the content is indexed the process that content goes through has to ensure that it is normalized and tokenization took place. Before content is tokenized the language of the content is identified and the normalization identifies accented characters and maps it into non-accented to enable phonetic searches. Tokenization process treats characters such as spaces, question marks, commas as word delimiters and splits the content into actual words that should be indexed and processed further. Lemmatization feature adds morphological and spelling variations to the content as well as misspelled or phonetic variants. This allows for search for all morphological and spelling variations as well as misspelled forms or phonetic variants to happen when people execute search queries. Phonetic search for names is available with SharePoint Search 2010 OOTB, more advanced linguistic features are available with FAST Search for SharePoint 2010.

There are several types of lemmatization:

1. Stemming - where the stemma is identified and applied to a word, ex: “doing” has “do” as it’s stemma

2. Lemma - where stemma of the word “worst” would be missed by stemming, but the lemma of the word as “bad” would be identified

3. Verbs - “greeting” may be a form of a noun or can have a verb “to greet” as it’s base, verb or noun dictionaries can provide the right level of lemmatization.

Lemmatization is language specific and allows to provide more relevant search result to end user and enhance the recall of search results.

Enjoy Smile