How to incorporate an Algorithm into a System using Tangram Pro

I created an AI algorithm that takes in images (jpeg) and determines whether the image is a lake or an ocean. The project, app, and requirements can be found here: https://github.com/jmstadt/fastai-v3. I want to be able to offer this app as a component to people leveraging a UAS. How could or would I leverage Tangram Pro to do this? What message or transform do I need to write (in Flex?) to create an interface (generic CSI?) to my component (algorithm)? How do I define messages that need to be used? (basically sending an image to the component and it returns a percentage likelihood that the image is a lake or an ocean). After I have the CSI and I do code gen in Tangram Pro, how do I then bundle the code gen, transform and my component and offer it to a UAS?

Oh my, that’s not going to be something you can accomplish over your lunch break.

  1. You’re storing your code on GitHub. That works in the general case but for your specific situation where you want your component and it’s repo to be available to everyone as part of the UAS demo system, it may need to be stored somewhere else, likely with other demo components. I’m guessing we have a spot in our GitLab instance for that, I’d talk to @fred.gillenwater , @matt.naveau , or @john.weis for more help there. I don’t know how we’re making demo code repositories available to all product users or where to store them for that purpose.

  2. I know we’re working on the Generic CSI capability. When that is done (and it might be, I’m not on that effort) you’d write a flex module which specifies the messages and data structures for your message set. That flex module would then be added to your instance of Tangram Pro by one of our programmers (if you were a customer with an on-prem installation) so the platform is aware of it. Then you could create a component with that message type and select the appropriate messages and call the code-gen capability on it to get a CSI for your application(s). Note that you cannot presently write flex modules in the platform or add them to the platform via the UI (unless a lot has changed!). You’d need to get in contact with @eric.zwirner and probably @ben.list to see how to write your module and then get it into the platform.

3.Two answers depending on what your “bundle” is intended for or in what sense you meant “UAS”.

3.A If your bundle is for the actual real-life (though imaginary) UAS, you’d take your bundle of code with some quality README.md instructions and work with the team that deploys code to the UAS to get it installed in the right spot and talking to everything. This is very situation dependent. Your bundle of code could be likened to my new bathtub, surround, bathroom vanity and toilet where you’re the Home Depot delivery truck. You gave me the resources, but installing them in my bathroom, (or putting the code on the actual UAS), is going to be a lot of manual labor because my bathroom (like most hardware configurations) is unique (unless my bathroom conforms to some universal standard such that robots can do the install for me.)

3.B If the UAS you referred to was the Tangram Pro system design of a UAS (The Treachery of Images is ever applicable) then you wouldn’t want to package the code-gen output with your AI code in the AI component repo. You’d create your AI component, specify it’s messages, connect it to your source code repo, and then add your component to the UAS system design. The CSI would then be generated anytime someone wanted it with the code-gen plugin. You don’t need to save the code-gen output in your AI repo.

3.Supplementary: As far as the transform, your system as I understand it doesn’t need a transform. You just have one message set with two messages, one that contains a picture and another with the percentage likelihood that it is a body of water. Transforms are only needed when translating data between different message sets. A transform could come into play if your message spec spoke to the larger UAS system and that system had it’s own message spec. If this was the case your transforming program would probably live on it’s own component that would serve to translate between your AI component and the rest of the system. Even so, you’d generate the transform via the Tangram Platform and would not generally store that code in the component repo itself. If this is the transform you were referring to, you would also need to write a transform function in flex and talk to @ben.list to get that added to the platform too. As with the generic-csi and the flex-modules, this cannot presently be done through the platform but must be accomplished in communication with Tangram programmers.

I hope I got everything and was able to shed at least some light on your series of questions.

2 Likes

Thanks @joe.richardson. This helps a lot! So, if I wrote this “component”, then within that component would I “call” the CSI generated by Tangram Pro. I.e. the CSI code would be a function or method that I would “import” into my component code for use?

@mark.stadtmueller yes you have the correct idea. Just a few notes: there’d be multiple function/methods (equivalent terms but java says “method” where C++ says function) in your CSI which you would use to create, send and receive your messages. It wouldn’t be just one. Depending on your language “import” may not be the right term but it’s the same idea. You’d import/include the generated CSI library and then use it’s functions/methods to do your message work. The CSI is kinda like all your vocal organs and your auditory senses and then your AI program is the brain, hearing through the ears and telling the mouth what to say, but unable to hear or speak on it’s own.

2 Likes

Thanks @joe.richardson, that really helps, I think I am starting to get this. Thanks also for pointing me to the correct vocabulary (I find using the correct vocabulary is a big step towards understanding). Thank you!

2 Likes