Many application development companies regadless of web development or windows development want or need to implement search functionality. However, it is a commonly underestimated function and it continues to evolve over time. Interestingly users want search to have minimal to no interaction while having a maximum result. With data and collection systems becoming more and more complex this becomes and increasingly difficult challenge. I remember a system I was architecting for a customer where the customer wanted to enter a DNA result that consisted of an 800 to 1600 character string into a web application and have it search a database using an algorithm providing scored search results. The customer was convinced that a basic desktop machine would act as a server and be able to conduct the search against a large database efficiently. The production architecture needed to support the customers’ performance requirements was a High Performance Computing hardware environment.  Like many customers they didn’t understand the complexity of certain functions. Thinking through this topic recently had me researching how functions in systems and their architectures evolved.

Architecting a system today has many facets, and search certainly is a prominent one. Searching for information is not a new concept but a heavily evolving one. Once computers evolved beyond just basic mathematics and started capturing, storing and manipulating other data the need for search began. Early systems collected data that was somewhat structured in files and databases. Search functions found data quickly within those structures. With the development of relational databases and more complex data capture search the tools for search had to grow. Also the acceptance and use of computers was growing and more and more.
Architecting search within a system has consistently had to recognize simultaneous evolutions. Database tools added the ability to index tables to help search perform better. Search appliances like Wizards emerged for more technically savvy users to pull data from a data source. Multiple levels of searching complexity were emerging. While these searches largely dealt with structured data stored in systems, at the same time this evolution was occurring what cannot be ignored is the emergence of the internet and its impact on search. 

Early on companies like Yahoo profited on the simple concept of locating content. While this wasn’t structure data as in databases internet standards of things like meta-tag’s and other items made it possible for users to find content early on. Searching on the internet allowed users to enter terms and content related to those terms would be returned. Later companies like Google would improve the algorithms and set that industries standard for a time. E-commerce companies were also integrating user shopper experiences with search as a means of driving revenue. So while a user shopped for shoes, related items and previous shopping items would appear in the links and advertising throughout the system. While the motives were different the capture of information and providing relevant data back is essentially an implied search. The evolution of the internet and its potential was impacting local systems.

Users’ expectations were changing as the interaction was to enter in a few terms and that brought back content they wanted to see. At the same time computers continued advancing in hardware and use. Pictures, Videos, art, music files evolved to become more common to be stored on systems. In fact digital has become so big that companies like Kodak have stopped producing film based cameras. Users have embraced and ran with the lower cost and portability of digital media. This new media has presented a new challenge and forced search to evolve in multiple ways again.

Architects and systems were faced with growing use for search.  Users were searching as an exploratory exercise as more complex data and more types of data were being captured. Allowing for the advancement of tools like Online Analytical Processing (OLAP) and reporting tools. Users weren’t looking for specific data as much as looking to see what trends might appear in the data. These tools while technically complex have easy to use interfaces that allow users to review and analyze data. The complexity lies in the architecture and backend. The emergence and development of these tools was a move from appliance parts of a system to search to a full blown system of its own.

Users now expect applications to be able to search both structured and unstructured data. They want to give as little information as possible and quickly find very relevant search results. Algorithms and techniques for searching continue to advance because they must--including incorporating e-commerce like changes in the system and having subtle changes help the customer get to the results they want more quickly. One of the many reasons unstructured data evolved was not only digital media but mobile devices.

This latest evolution has occurred simultaneously with the acceptance of mobile devices. Now users have a high level of portability and connectivity to data. These mobile tools work quickly using touch screen technology and other key changes that impact the user experience for working with data. This has resulted in a need for better performance and system architectures that incorporate different devices, connectivity, and desired results.

Today’s cutting edge searches involve grabbing information from a part of a picture and searching for related information. Searches that work from audio files or live audio and provide related information quickly on portable devices is another technology that has been developed. Users want more with less required of them, resulting in more complex algorithms and models for searching.

Successfully architecting a system means taking a lot of factors into consideration. A successful solution can't overlook what the implementation's search functionality has within an enterprise system. Architecting search as a part of a system today means taking many factors into account. Understanding the user’s expectations and desired results has become critical to the successful use of a system. What devices are targeted for use, what is the complexity of the data, what type of data, and other questions like these are all key to get answered to develop a successful search system. Working with customers to identify the business rules that lead to implicit and explicit searches is important as systems more and more are expected to show relevant data.