The project goal – establishing Scalable Cloud Web Infrastructure for AIP search engine based on Amazon EC2 Cloud Computing platform. The purpose is to enable AIP with access to the global Web audience to take advantage of human intelligence and users' feedback in order to improve AIP search engine queries' quality results. As AIP sophisticated search engine is highly CPU consuming, the Scalable Cloud Web Infrastructure will automatically deploy additional computing resources, when demand in searches is growing and decommission the mention resources, when demand for searches goes down.
On one side, it will provide unlimited scalability – Cloud Web Infrastructure will be able to satisfy unlimited quantity of search requests deploying additional Cloud resources, on the other side – it will shrink the use of resources to minimum, when demand diminishes.
Automatic resources scale-out and scale-in are the major requisites of the Web service targeted to success, as many potentially successful Internet ventures suffered heavily from inability to scale due to non-scalable classic Web infrastructure.
Due to highly CPU consuming nature of artificial intelligence algorithms AIP develops, the proposed Cloud Web Infrastructure should be implemented using Amazon EC2 Cloud Computing platform in order to provide unlimited scalability and satisfy unlimited quantity of requests by multiple users simultaneously using multiple Amazon EC2 instances of the Scalable Web Infrastructure components started on demand. To improve performance and security and decrease ongoing expenses, the recommended Operating System for implementation is Linux.
The project includes:
1. Components choice, development and integration of Scalable Cloud Web Infrastructure
Scalable Cloud Web Infrastructure to be developed specifically for AIP requirements, includes the following components to be developed according to the following project stages:
1.1. Development of multi-user host access to the existing application prototype.
1.2. Choice of solution third-party components, including testing, benchmarking and analysis, choice of servers HW, including highly scalable and fast storage platform, analysis of optimal CPU, RAM size, choice of virtualization technology.
1. 3. Choice of Distributed File System, deployment, testing, benchmarking and customization
1.4. Deployment of Queue Server/s to enable highly parallel and scalable distributed queries processing.
1.5. Integration of patent-pending IndexLayers highly scalable distributed archiving capabilities (to replicate and distribute the queries between multiple copies of indexes, to accumulate queries history, to search archived queries)
1.6. To architect and develop distributed objects IndexLayers Metadata database
1.7. To provide AIP Search Engine architectural recommendations for the dynamic Cloud environment (to accomplish - responsibility of AIP)
1.8. Integration of AIP search engine with the Cloud Web Infrastructure
1.9. Web Servers choice, deployment, customization
1.10. Application Servers choice, deployment, customization
1.11. Web Load Balancers choice, deployment, customization
1.12. Develop Web GUI interface
1.13. Integration of all mentioned above components
1.14. Monitoring agent/s
1.15. Development Cloud Management system, including automatic resource allocation/decommissioning mechanisms
1.16. Central monitoring system
1.17. Develop storage replication mechanism and integrate it into Cloud Management system
1.18. Develop Cloud Backup mechanism using Amazon S3 storage
1.19. Development of packaged OS images - Amazon AMIs per each static and scalable system component
1.20. DR architecture development including remote data replication
1.21. Development of secure remote management
1.22. Choice and integration of reliable and highly scalable universal multilingual billing platform, including customization.
1.23. Development of patent landscaping visualization on the basis of metrics calculated and supplied by AIP algorithm per patent