-
Notifications
You must be signed in to change notification settings - Fork 0
general purpose web crawler for UNIX systems using PosgresQL backemd
License
azzmosphere/azzmos
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
AZZMOS PROJECT
==============
VERSION 0.1
Azzmos is WIP project that attempts to build a multi-threaded web crawler that can use different
index algoritms. The aim of the project is to provide three programs on completion:
These are:
azzmos - web crawler.
azidx - indexer
azctl - controller function.
The program requires a running and multi-threaded version of postgresQL database.
COMPILING POSTGRESQL
====================
postgresql will need to be compiled in the following way:
./configure --enable-thread-safety
make
make install
In the postgresql.conf file the following should be added:
client_encoding = iso-8859-1
INSTALLATION INSTRUCTIONS
=========================
Ensure that psotgresql is installed and running.
Modify the ddl/install.sh script.
Create a build directory 'mkdir build'
Change directory to build, 'cd build'
Use the following commands:
../configure
make
make check
make install
PREREQUISITS
============
To install azzmos the following is required:
PostgresQL compiled to run in multi-threaded mode (v 8.2.16)
cURL development headers
PCRE library and headers
About
general purpose web crawler for UNIX systems using PosgresQL backemd
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published