The AyeBee Event System is a set of programs, services and ActiveX components for Windows that allow the creation, routing and subscribed consumption of data events, in a real-time environment.
The tools are intended to work in parallel with existing or new systems, and a graphical component (ActivePad) is provided to display continuously-updating real-time data within web pages.
Currently, the system is designed to be used within intranets because its non-standard communications are likely to be blocked by firewalls. An enhancement may be considered to allow it to use 'continuous HTTP' to work through firewalls.
Data events can be text messages, new values of a data field (such as a stock price), news events, job progress information, or any other packet of data that can be logically displayed or used by the system.
They are transitory pieces of information that relate to the time at which they are created and, while the data on which they are based may be held in a permanent repository such as a relational database, this system is designed to use them as they occur. As such, it provides no method for persistence of that information within its infrastructure.
Generally-speaking, if a client is not currently requesting events relating to a specific item of data, then it will miss them. This doesn't stop a data source from sending past updates when a client first subscribes, and the message protocol will allow up to a 100 numeric values to be sent if the wish is to initially populate a tick graph.
A latest stock price, where the current value is overwitten by any change. Whereas the system does not retain previous values as such, one of the graphic components provided will buffer and display up to the last 100 values within a tick graph.
Job progress, where a supplied graphic component can display a progress message and/or percentage in the form of a progress bar.
A news event, where a supplied graphic component can display a scrolling news ticker, including active hyperlinks. These can be triggered by database events (such as a large order) with, say, an event-supplied hyperlink bringing up the relevant details from a corporate intranet system.
Data events are identified in a similar way to Microsoft DDE link items, and a DDE server for creating live data links within products such as Excel is provided.
An item is identified by a Topic Name and Item Name.
A topic usually relates to the source and type of the data, and each topic is normally only provided by one specific program.
A topic may be machine independent (e.g. A database server, or multiple providers of data for a news ticker), or machine-dependent (e.g. machine performance statistics relating to the machine on which the data provider is run).
If the data provider is considered to be machine-dependent, then the recipient of the event will specify the source computer within the data request.
If the data provider is considered to be machine-independant, then the recipient of the event will not specify the source computer and, if that topic has multiple sources, the events will be interspersed according to the timeline.
An item relates to a specific piece of data within a topic, and the name only needs to be unique within that topic.
Examples would be the identity of the stock (e.g. 'MFST') within a 'STOCKPRICE' topic, or a specific performance item within a 'PERFMON' topic from a specified computer.
Item names can be reasonably complex and, if the appropriate data source for that topic allows it, could be 'soft'. An example would be the use of a simple SQL statement as the item name, which could then be interpreted by an intelligent topic server as a request to execute the SQL statement and return the results as a live event, on a periodic basis. This is not the recommended way of triggering data events within this system since database triggers are far more efficient and timely, but it shows the flexibility of item names.
Data events are created by Data Providers, distributed by a central Data Router, and requested by Data Clients.
Data Providers are programs, services or database processes that send event data upon request to the central Data Router.
The AyeBee Event System currently provides a pre-built service for transmitting computer performance data, ActiveX components for managing data subscriptions and unsolicited data, and a subscription-monitoring service and extended stored procedures for use within database triggers.
Static C++ or C# libraries may be released in the future to allow for the construction of data providing services, without the overheads of ActiveX.
An ActiveX component is provided that allows the developer to create their own subscribed topic server, and that manages
the subscriptions on their behalf.
The component monitors subscriptions for one or more topics and can transmit events on demand, or use an internal polling timer to request data on a routine basis.
A simple ActiveX component is provided for transmitting unsolicited events on demand.
Unsolicited events should not be used where the number of supported items is high, and the interest in these events is relatively low. In those cases, a subscription-based model should be used.
A service is provided that manages subscriptions to machine performance data.
Any performance data that is available to the PERFMON program is available to this service. This can include database performance statistics, as well as process, memory and network statistics.
Data is updated on a one-second basis, and is suitable for displaying in tick graphs.
An extended stored procedure is provided for use within Microsoft SQL Server database triggers that can transmit data change events.
A separate service is provided that allows another stored procedure within the trigger to check whether any relevant subscriptions exist before performing any extensive database processing.
If the service isn't used, the events will be treated as unsolicited. i.e. They will be sent whether subscriptions exist, or not.
There is a possibility that static libraries for directly linking into developer's own service programs might be released in the future.
This will not be possible in the first release.
The central Data Router is a service that manages data subscriptions from clients, consolidates them into single requests to data providers, and distributes the event data back to clients when received from the data providers.
There is a potential within the architecture to use multiple routers (on separate machines), in either a peer-based or hierarchical-based structure.
The initial model uses one Data Router service that talks directly with each Data Provider, and with all of the web-based clients .
Peer-based routers would independently subscribe to the Data Providers, and would keep their own collection of web-based Data Clients.
Hierarchical routers would be formed in a pyramid, where the lower-level routers would subscribe to a parent router rather than the Data Providers, and only the bottom layer of routers would manage subscriptions from Data Clients.
Only the top-level router would need to subscribe to the Data Providers.
This scenario would provide the best way of spreading high workloads, and reducing the load on the Data Providers.
The clients are ActiveX components for use within programs or web pages, and a 'systray' program that can provide popup notifications, and that also acts as a DDE server for DDE-aware programs such as Excel.
A mail service is also provided that will send an email to an SMTP server when it receives messages on the appropriate topic/item, such as from a database trigger.
The ActiveX component for use within web pages provides a graphic interface that can not only can display active data in various graphic forms, but that can also allow the end-user to edit the graphic interface within the web-page.
The current web component is mainly designed for use within Intranets, since it uses unicast datagrams over non-standard UDP ports that would normally be blocked by corporate firewalls.
An ActiveX component is provided that allows a client program to subscribe to various topic/items, and that returns program events when data events are received.
An ActiveX component is provided that allows a web developer to include real-time updates within the current page, and to display these in graphical form.
The graphical content and layout is defined via server-side templates, but is uniquely editable within the client-side browser.
The component is called ActivePad, and is separately documented via the left-hand menu.
A 'systray' program is provided that allows message-type events to be displayed in a popup desktop window.
These notifications appear in a slide-up window that either times-out, or disappears when clicked upon.
The user can select to receive events for fixed topics according to his computer-name, his logged-in user name, and/or multiple user groups.
Message-defined buttons can be displayed in the window, such that clicking on a button can send a predefined follow-up message.
Although this can be used for a simple question and answer, the fact that the next message automatically replaces any notification that is currently displayed means that it is not a safe process for workflow processing.
The 'systray' program that can be used to popup messages on the user desktop can also be used as a DDE server.
A built-in function assists with the creation of the appropriate cell formula for Excel and, once pasted into a spreadsheet, will automatically subscribe and continuously-update the item when the saved spreadsheet is next opened.
A service is provided that answers to a fixed topic, and will send emails with the specified content and header details to an SMTP server.
This allows any Data Provider to trigger an email.
It is not recommended for bulk email since it buffers events in memory, and these could be lost in the case of a process or hardware failure.
RSS/XML Feeds might be implemented in the future, both as an RSS server in the form of a service that acts as an Event Data
Client, and as an RSS client in the form of a component that can be used in an Event Data Provider.
The physical and logical transport protocols are designed to handle high-volume data in a timely and efficient manner.
To enable high performance, the system uses connectionless UDP datagrams with a mixture of unicasts and multicasts.
Unicasts have a single destination, while multicasts can be read by any network node that subscribes to a particular
multicast group.
Most events use directed unicasts, but multicast is useful for such tasks as a Data Router discovering which Data Providers
are active and supporting a requested Data Topic.
Since UDP bypasses the handshaking involved in the use of TCP, it enables a significantly faster throughput and, with
modern networks, proves no less reliable.
The system allows for multiple discrete network zones that operate without any danger of overlap within the connectionless
environment. Any one machine can only operate within one zone, although a web client can use multiple zones, according to
the the zone in which the individual web servers operate.
Examples of different network zones would be Live, QA, and Development.
The transmitted data within each zone can be encrypted if so desired, and the installation of machines into individual
zones can be password controlled.
As a closed system, data is transmitted using a compact streaming protocol to avoid the performance overheads associated
with industry-compatible interchange protocols such as XML, etc.
To handle the high-speed connectionless UDP protocol, the processes use priority-driven, cross-thread FIFO buffers to
assure timely and safe processing of all events.
Real-Time means that data is transmitted from the data provider to the data client within a fraction of a second.
As to whether the data is received immediately upon the change of the source data will depend upon the method used by the data provider to monitor the source of the data.
The use of extended stored procedures within triggers in Microsoft SQL Server allow the event to be triggered immediately.
Currently, the only method available for Oracle databases is to use Oracle Queues. The internal mechanism of Oracle Queues (within Oracle 9i at least) appears to use a one second poll, so there could be a slight delay.
The supplied service for performance monitoring uses the same data as used by the Perfmon program, and use a one second interval for convenient display within tick graphs.
If the ActiveX components are used for event creation, it is up to the developer as to how the events are created and transmitted. However, there is a built-in polling timer that can be used to request data for subscribed items on a periodic basis. This is the easiest method, since the developer does not need to monitor which items are subscribed, and just needs to respond to the individual requests for data.
The distribution of data within this system is completely interrupt-driven using dedicated threads, and there are no artificial delays induced by polling intervals.
Data events may be sent by a data provider on a subscription basis, or be sent unsolicited.
With subscribed events, the central data router service keeps a note of the clients who request a specific item of data, and notify the data provider whenever the first client subscribes, or the last client unsubscribes.
A well-written data provider will monitor these events, and only send updates to the central router when it knows that clients are currently interested.
The data provider is not aware of the multiple clients, and it is the central router that distributes each update to the relevant client.
With unsolicited events, the data provider does not bother to monitor the data requests from the central data router, and it sends the data events periodically.
This makes it simpler to write the data provider but, because of its relative inefficiency, it should not be used for data providers with a large number of potential data items but a relatively-low subscription level.
Although the data provider sees these items as unsolicited, the central data router only ever sends data to clients who have subscribed to the data item.
The system should not be used for process-critical applications.
While the system is very reliable and does not appear to lose any messages, it uses high-performance datagram streaming protocols that intentionally do not use handshaking to ensure message receipt.
To test for the loss of messages, there is a system topic/item provided that will stream sequential updates at various speeds, and will display any losses, or multiple or non-sequential receipts. So far the only losses were via a dial-up modem where the flow of data exceeded the line capacity.
However, because it uses connectionless protocols, the sender will not know whether the intended recipient(s) have been listening or not, unless the developer adds their own high-level handshaking protocol.
A series of cross-thread FIFO buffers are used to capture and store received data, and separate buffers can be used to push through high-priority events ahead of low-priority events.
While the client components can receive and buffer a million or so small data updates while awaiting user processing, if the client process isn't running the data updates will disappear into the ether.
Irrespective of the intended reliability of the system, the author will not accept responsibility for any losses due to the non-processing of events.
The current system is designed for the Windows architecture.
The server-based programs are designed to run on Windows NT, 2000 and XP as installed services.
The current developer components are ActiveX, and so can run on both Windows server and workstation operating systems.
The sys-tray component for popup notifications, and for DDE program support, can run on both Windows server and workstation operating systems.
The ActivePad web-page component can run in any browser, and on any operating system, that supports ActiveX components.
For future compatibility, the DDE server will be converted to use OLE links instead.
The current system is completely written in C++ and, for developer interfaces only, uses ActiveX.
The system is currently being ported to .NET managed-code using C#.
A Java applet-based component is currently being considered as an alternative to the ActiveX web-page graphic component.