They say that necessity is the mother of invention and it happened to us. While doing some internal projects in house we needed access to some external 3rd Party API’s. Many of the fastest and easiest to implement API examples were in PHP. So it was only natural that engineering would want to add PHP to the list of languages that we support in Conductor.
By the time our services team was finished testing the API integrations (about a day), the engineering team had implemented the PHP support. This was great news for the services team as they were now able to use Conductor’s Input and Unify Wizards to create a complete solution. First to gather data via the API calls from external sources on a recurring/scheduled basis and have the extracts automatically detected and ingested into native database tables. Then data was unified and cleansed into a data model using the Unify Wizard (see video). The final unified data was then presented using BI visualization tools.
The next paragraph is the well thought out description of Conductor, but what it really is to me is a developer’s dream. Many of us have used the open source and madly expensive tools that perform data integration and management but they make it like pushing a rock uphill. They tell you no coding or scripting is involved and then tie your hands with popups and boxes and requirements that don’t let you do anything functional like “making a simple file copy.” Oh you can do it, but then you have to add another little box to the diagram and fill out 3 fields…oh wait, how do I get my filename from the previous step? Go add one of these things to the diagram…you get the point. Conductor makes the hard things easy, we have Wizards that can reduce or eliminate coding on things like file ingest, file output, data unification and more. However, we let you code when you need to, and when you need to, you have a choice of 13 languages to choose from. Choose the best language for you, when you’re happy, so are we.
The Conductor software suite provides a rich and powerful, heterogeneous, multi-node, job scheduling/workload automation and workflow management orchestration engine to drive big data ingestion, transformation processing, analytic modeling and reporting workloads. Conductor can be configured for a single node, conventional database warehouse to an enterprise class, multi-node system. It has a proven track record for workload automation on petabyte sized system and tens to hundreds of billions of records/events per day utilizing MPP Databases and Hadoop clusters, yet it is nimble enough to run in single node and smaller environments. Conductor is included in our all-in-one cloud based data management platform : CloudCella.