The Conductor software suite provides a rich and powerful, heterogeneous, multi-node, job scheduling/workload automation and workflow management orchestration engine to drive big data ingestion, transformation processing, analytic modeling and reporting workloads. Conductor can be configured for a single node, conventional database warehouse to an enterprise class, multi-node system. It has a proven track record for workload automation on petabyte sized system and tens to hundreds of billions of records/events per day utilizing MPP Databases and Hadoop clusters, yet it is nimble enough to run in single node and smaller environments.
Easily design, publish and re-use workflow tasks to not only create transforms, but just about any code you can dream of. Conductor doesn't get in your way by forcing you to fill out popup after popup just to run your code. Write your code how you want in the language that you want. We don't get in the way, but we can help with ton of cool features like parameter substitution, resource locks and more. Tasks can be re-used and run across hosts and even different OS types. This not only saves you time but money! Our parameter substitution ensures that you're not shoe-horned into creating "almost" the same code over and over again, pass variables, names, file names, IDs and built-ins across tasks. Set variables at low level or leave them blank to be filled in at run time. The control is in your hands, this is the kind of control you've dreamed of. If you've been shackled with the likes of SSIS, ODI or any of those other tools, try Conductor and free yourself. Our Wizard Factory helps to reduce and even eliminate coding,complexity and endless tabs and clicks. Our Wizards put big data in the hands of the people that use it, so you don't have to be a coding expert. If you want one of the best, easiest to use job/process control and workflow management solutions for your ETL or ELT, take a look at Conductor (there's even a free trial).
The Conductor Desktop application provides desktop/laptop Windows users the ability to use, manage and administer the Conductor system through a powerful, remote, graphical user interface.
Conductor is included as part of our CloudCella offerings. No hardware to buy, no database to buy, no BI tool to buy, no integration required, no operational support required just load your data.
Conductor can be purchased as an on-premise solution. You provide the hardware, we provide the tool.
Ready to Get Started?
Conductor is a software solution comprised of the following primary installable software packages:
- Conductor Server
- Conductor Worker
- Conductor Desktop
- Wizard Factory
The Server package provides a rich and powerful, heterogeneous, multi-node, scheduling, process control and workflow management and orchestration engine to drive big data ingestion, processing, analytic modeling and reporting workloads. The system is dynamically scalable from a single node, conventional database warehouse to an enterprise class, multi-node system, well suited for petabyte data management and tens of billions of records/events per day utilizing MPP Databases and Hadoop clusters. Server includes a locally installed version of the desktop application which provides the ability to use, manage and administer the Conductor system through a powerful, graphical user interface run directly on the server.
Conductor User Interface & Tools
The desktop is the main navigation interface for process control and workflow management as well as a host of other tools, wizards and utilities. A simple menu driven system launches a tabbed interface of utilities much like a modern browser.
A Full Featured Tool
- Detailed User Permissions
- Editors with Intellisense
- Reporting and integration with third party BI Tools like Tableau, Microstrategy and Cognos
- System Messages
- Query Builder with intuitive and extremely powerful variable prompting and substitution
- All of the Wizard Factory Components (Input, Output, Unify, Data Lifetime, Content Release)
Supports a Ton of Databases
- DB Independence and optimized load processing support for Teradata, Netezza, SQL Server, Hive, Red Shift, Greenplum
- General support for any database which uses ODBC/OleDB
- Workflow Tasks written in can use database agnostic functions to make single code line work in any supported DB environment
- Application implementer has the option to code rules / transformations using stored procedures, stored functions, views, static SQL or dynamic SQL from a Task in real time
- DBAgnostic Metadata API provides consistent, single code line management of DB objects across supported DBs (table exists, table describe, script DDL, etc)
- Centralized connection management
Sophisticated Process Automation and Workflow Management
- Tackle your heaviest system tasks with Conductor's powerful job scheduling and workload automation capabilities.
- Easily integrate existing command line executables and capture their output in a centralized easy to use log and process viewer
- Create new tasks in the language of your choice using clean visual editors with intellisense, even mix and match languages between tasks of a workflow.
- Easily execute tasks on a specific node or groups of nodes.
- Add or remove Conductor (machine) nodes in a few minutes. Mix and match Windows/Linux nodes to perform the processing you need.
- Plug and play : use your existing hardware and infrastructure. Add a host as a Conductor node with a few clicks.
- Full distributed process control, task creation and workflow editing.
- Full version control on tasks and workflows allows "in production development" to be performed without risk to production processing.
- Parameter substitution on workflow and task level provides the ability to pass information from task to task in a workflow. Feed one task with information from another, even across nodes and technologies (windows to linux etc).
- Dersigned to be reused, the parameter substitution technology enables you to write generic tasks that can be reused across workflows. Parameter substitution works across all supported languages.
- Graphical workflow and task design. Plus a powerful and flexible visual scheduler.
- Integrated Resource Control an be used to throttle/mutex application logic across tasks and hosts.
End to End Data Management Accountability
- ETL right out of the box.
- Unique record level tracking on all data sources.
- Automated data lifetime purging and database maintenance.
- Full detailed tracking throughout the data lifetime from ingest to reporting level.
- Duplicate file detection.
- Built in smart parsers for almost any ASCII data feed.
- Loading paradigm built on reusability and production scenarios.
- Automatically manages LandingZone, new data detection, prioritized data processing and data decompression.
Unlimited Sizing with Multi-Node Scalability
- Supports High Availability Failover Clustering
- Add additional worker nodes as needed (horizontal linear scaling) even with heterogeneous operating systems.
- Replace/upgrade existing worker nodes one at a time or en-masse as desired (vertical scaling)
- Centralized logging, system monitoring, and error management. No need to log into multiple hosts to find out what's going on.
- Automatic feed processing balances load among assigned nodes.
- Proven production systems with volumes up to 100 billion records and 1 million files per day
Conductor User Interface
Find out more about what Conductor looks like and how it works.