Several odbc driver improvements.

  • In previous versions the number of simultaneously running queries never actually reached the 'query set max runnable' parameter value.
  • The issue related to adding extra characters after non-ASCII rows has been fixed.
  • Other minor bug fixes.

Several feature improvements and bug fixes.

GreenwichServer ODBC driver now can return results in wchar strings.

  • The Unicode support is helpful for those using perl-DBD-ODBC with GreenwichServer ODBC driver. since Red Hat supplies the version 1.50-3 built with Unicode by default.

Long restart issue has been fixed:

  • In previous versions there was a case when a full table verification process started even if the GrenwichServer has been shutdown correctly.
  • Table verification time after correct GreenwichServer shutdown has been reduced.

Partition corruption in specific condition has been fixed:

  • Partition corruption has been fixed for other uncommon conditions (part of this problem has been fixed in the previos version).

A problem in how the count(*) query results are represented has been fixed:

  • In previous versions there was a case of incorrect empty query results prompting.

An issue related to corruption of index partitions has been fixed:

  • In previous versions there was a case of partitions getting damaged in certain highly uncommon conditions.

Time enhancements:

  • ICU time has been replaced with unix system time. This enhancement allows GreenwichServer to work with datetime and time zones more strictly.

Partition elimination logic has been enhanced:

  • min/max verification has been optimized. This enhancement increases query performance when a lot of records are returned.
  • Incorrect partition skipping issue has been fixed.

A minor bug of processing VARCHAR and BINARY data types has been fixed.

Loading performance has been increased:

  • Loading performance has been increased by optimizing LoadBuffer internal logic. LoadBuffer consists of a reader that reads data from files into internal buffers and a writer that writes data from internal buffers into an index. Readers and writers operate in separate threads. Prior to the current version, the number of internal buffers was fixed and size was changeable via configuration file. In the current version number of the internal buffers is dynamic and increases automatically to achieve optimal performance.
  • This enhancement increased loading performance by over 30%.

Index caching logic has been fully rethought and enhanced:

  • The memory consumption is now fully predictable. With IndexCacheMode set to None queries consume almost no memory. With IndexCacheMode set to Full the IndexCacheSize parameter now precisely restricts the memory usage. So taking in account the memory used for load buffers and repository cache (no more than several gigabytes) one can accurately predict overall memory used by the server.
  • There is no performance degradation compared to the previous version.

Partition elimination logic has been improved:

  • For each column in a (compacted) partition, an index stores a range of minimum and maximum values that column contains. When searching a partition with a predicate on a particular column, the Core can quickly check if the predicate falls within the range of values held in that partition. If the predicate is outside that range, then no matching records will be found, and the partition will be skipped entirely for the search.
  • This enhancement increases query performance when few records are returned and most query time is wasted on indices scanning.

Volume based licensing has been implemented:

  • Two license types have been added: License limited by the overall data records number. License limited by the overall data size. If one of the limitation has been exceeded an error will be returned while querying against the GreenwichServer (file indexing will not be stopped).
  • The following commands have been added to the AdminClient: server [show] limits – to show limits and current value. server update limits – to update the current value. server set limits max – to increase limits by 20% and ask support about the license extension or delete old stripes.
  • If the license is exhausted, there are 3 possibilities: Buy license with greater amount. Delete old stripes. Temporarily increase limits by 20%.
  • Server update limits command runs automatically every 24 hours.

License files now support Windows CR+LF line endings.

The following functions have been added to the ASCIIFileReader:

  • Uppercase
  • Lowercase
  • Truncate
  • Substitute

The following configuration parameters have been deleted from the GreenwichServer.conf file:

  • MaxParallelQuery - performance tests show that this option is not of much practical benefit, and when set to a high value actually reduces query performance.
  • CostPartitionsNumber - became obsolete due to enhanced memory management logic.
  • IndexCacheRangeFraction, IndexCacheStrict - became obsolete due to enhanced memory management logic.
  • ServerLogging - there are no known conditions not to write the server log file.
  • LogEncoding - log files are now written in the user locale.
  • Security - there was only one option ('Internal') for this parameter.

The following tools have been deleted from the GreenwichServer distribution:

  • Compactor - duplicates compacting logic in server, but without modifying the repository - not useful.
  • Preloader - duplicates 'schedule preload' command in AdminClient.

Query log information has been improved:

  • In the older versions query sent against GreenwichServer from Oracle using odbc was logged in the .log only after the next query. Now every query run against GreenwichServer from Oracle using odbc will be logged right after the results are returned.
    • The inforamtion about optimiser choice has been added to the query log.
    • 1 : OPTIMISER CHOOSING BETWEEN 2 INDEXES:
    • 1 : DATA_IDX1
    • 1 : DATA_IDX7
      The inforamtion about partition elimination using new min/max has been also added to the query log.
    • Fetching from index is finished, total partitions: , skipped by min/max:

Query kill logic has been enhanced:

  • In the older versions a query could only be killed on the partition scanning phase using 'query kill all' command. Now it is possible to kill query on any phase using 'query kill '.

No such loader issue resolved:

  • It is possible to view loader information in any case. Previously there was a problem when in some conditions it was impossible to view information about a loader by its number.

Ability to add Schedule priority in DiscoveryClient has been added:

  • Now it is possible to set schedule priority using flag -o from DiscoveryClient.

Assertion failed error on the date formats with BINARY type has been resolved:

  • Assertion failed error while using user roles has been solved. Now it does not matter how many columns a user has access to.

Fixed issue with Greenwich crashing when 'where' clause is used in queries against Greenwich system tables made via QueryClient.

Issue with deleting old index partitions while querying against them has been finally fixed:

  • In the previous versions of GreenwichServer, if a partition gets deleted by the DefaultMaxStripeAge job while there is a query being executed against that partition, the query would fail and the user would receive an error. The new version will allow the query to finish and a message will be logged into GreenwichServer.log.

GreenwichServer memory consumption has been improved:

  • The GreenwichServer instance with approximately 10 partititions closed every day per index will reduce RSS memory usage 5-7% of every query type towards the previous version of GreenwichServer without degradation in query performance.

Known issues:

  • Using 'where' clause in queries against Greenwich system tables made via QueryClient may lead to server crash.

CostPartitionsNumber parameter has been added:

  • Every query is executed against a single index, which is selected by calculating a cost for every index in a table (costing). This parameter restricts the number of partitions to be costed for every index. Such restriction leads to the RSS memory reduction and query performance acceleration in a large GreenwichServer instance, but in rare cases can lead to wrong index selection.
  • The GreenwichServer instance with approximately 10 partititions closed every day per index will reduce RSS memory usage 4 times towards the previous version of GreenwichServer without degradation in query performance.
  • CostPartitionsNumber 0 means that all partitions will be costed for every index in a table. Behavior identical to the previous versions.

Support of system Rhel6 Openssl 1.0.1 version has been added.

Issue with deleting old index partitions while querying against them has been fixed:

  • In the previous versions of GreenwichServer, if a partition gets deleted by the DefaultMaxStripeAge job while there is a query being executed against that partition, the query would fail and the user would receive an error. The new version will allow the query to finish and a message will be logged into GreenwichServer.log.

String to timestamp convertion for Oracle has been added:

  • Now GreenwichServer can parse a string containing date, converting it to date format.
  • It is possible to query against GreenwichServer using one of the following syntax: {ts'2010-04-21 04:26:01'} or '2010-04-21 04:26:01'.

Federator data parameter passing from Oracle DB using ODBC has been improved:

  • From the version 3.5.4 Federator did not know how to handle data parameter sent from Oracle DB. In version 3.5.5 this feature has been added back again.

Query and Discovery Clients enhancements:

  • There were some conditions in which old clients did not call disconnect() while task is already done. That led to client port becoming unavailable and "M00088-Could not talk to client: Connection error" error in the log file. This problem has been solved by rewriting that part of the code.

Index behaviour with the null index fields has been improved for both ASCII and Unicode cartridges:

  • In the older version if the index field is null, this field will be indexed in every case. In the newer version this behavior was changed to the opposite, now if the index field is null, this field will not be indexed. To achieve behavior of the older version, user should set default configuration parameter in the cartridge configuration file like the following: default "", where field_N is the field number desired to be indexed in every case. If your system has some null index fields, be sure to set default parameter.

Csync has been accelerated:

  • Files could be copied remotely faster because of scp carried out from the search file loop

Stripe switching enhancement:

  • two new indexes on repository were made: CREATE INDEX AA_FILE_HISTORY_STRIPE_NO_IDX ON AA_FILE_HISTORY (STRIPE_NO); CREATE INDEX AA_FILE_STATUS_IDX ON AA_FILES (STATUS);
  • one index on repository was deleted: DROP INDEX AA_FILE_HISTORY_TABLE_NAME_IDX;

Core index cache size management problem has been solved:

  • IndexCacheStrict parameter has been added. It helps to use new versions of cost, scan and fetch core functions.
  • Memory usage has been reduced and has been made more predictable.

SQLite cache has been improved:

  • 50% of repository cached only if this part is less than 1Gb
  • _RepositoryCacheLimit parameter could change 1Gb to some value

Federator has been changed:

  • Data will be returned as soon as Federator got them
  • Query kill logic in Federator has been added. Now it is possible to kill query from the Federator
  • There is no cache in Federator anymore

BINARY type data representation enhancements:

  • Aggregate functions now work correctly with BINARY type
  • GROUP BY works correctly with BINARY type
  • UNION works correctly with BINARY type
  • Incorrect lead symbols have been deleted from the results

Deprecate IndexMergeDepth parameter from Greenwich:

  • There was a time where filtering was performed on the client side, thus there were situations, in particular if a query contained multiple predicates but each predicate field related to a different index, when a lot of data had to be retrieved across the network and filtered in the query application. MergeDepth provided a workaround by using multiple indexes to fetch and merge the results on the server-side thus reducing the amount of network traffic. Nowadays, all filtering is handled in the client so the MergeDepth? parameter provides no performance benefit in any situation.

Index behaviour with the null index fields has been improved for ASCII cartridge:

  • In the older version if the index field is null, this field will be indexed in every case. In the newer version this behavior was changed to the opposite, now if the index field is null, this field will not be indexed. To achieve behavior of the older version, user should set default configuration parameter in the cartridge configuration file like the following: default "", where field_N is the field number desired to be indexed in every case. If your system has some null index fields, be sure to set default parameter.

The new Federator has been implemented for the Greenwich Server 3.5:

  • BINARY type support has been added.
  • TMaxRunnableQueries parameter and the number of the currently running Greenwich Server queries are analyzing in a BALANCE set type. Now the new Federator in the BALANCE set type will check the difference between MaxRunnableClients parameter and currently runnable queries, then send the query to the Greenwich Server with the lowest one.
  • Standby and attempts logic (MaxAttemptsCount and ConnectTimeout) has been added. Now the new Federator within the share all set types will try to send a query to one of the Greenwich Servers, if something wrong try another one. If there are no connections to every Greenwich Server from the list, Federator will try to connect MaxAttemptsCount times with the ConnectTimeout timeout in seconds. Within the share nothing (PARALLEL) set type Federator will try to connect only MaxAttemptsCount times. If the MaxAttemptsCount = 0 - try in a loop while user does not stop the connection.

Standart cartridges such as "Unicode" and "ASCII" have been improved

REINDEX and VACUUM commands have been added to the AdminClient. There is no need to stop the GreenwichServer before them anymore.

Standart cartridges support has been implemented:

  • BINARY type support has been added.

'table column modify binary' command has been removed.

SQL statements enhancement for the BINARY data:

  • LIKE statement enhancement.
  • UNION statement enhancement.

Greenwich Server Documentation enhancements.

ODBC and JDBC drivers have been improved.

Standart cartridges such as "Unicode" and "ASCII" have been improved:

  • BINARY type support has been added.

Greenwich Server Documentation enhancements.

AdminClient enhancement for the binary type data:

  • 'table column add binary' has been added.
  • 'table column modify binary' has been added.

SQL statements enhancement for the BINARY data:

  • SELECT DISTINCT statement enhancement.
  • MIN/MAX statement enhancement.
  • GROUP BY statement enhancement.
  • HAVING statement enhancement.

The new BINARY data type has been implemented:

  • Keeps data in indexes in a more compact form. BINARY type supports data length from 1 to 255 bytes, now User may choose the data length itself. For example in previous Greenwich Server versions IPv4 format has held 8 bytes, within the NUMBER type. Within the BINARY type IPv4 format will hold only 4 bytes. IPv6 has been held only by VARCHAR type 32 bytes, now it will hold only 16 bytes. So binary type is a more compact type.

Fixed length BINARY fields (1, 2, 4, 8 bytes) represented as integer numbers.

The new cartridge interface has been added:

  • The new BINARY data type support has been added.
  • Outdated methods have been removed

The new BINARY data type support has been added to the ODBC and JDBC drivers.

The three digit Greenwich Server product version transition:

  • Three digit numeration meaning: "Major version", "Minor version", "Patchset".