Tutorial Talend Component Kit 6: Implementing "List" Input and further Output Processing

The purpose of a component of the "Talend Open Studio for Data Integration" is to relieve the user of as much work as possible through simple input. The Jira input component implented in the last five tutorials does not provide much functionality yet. Only the output raw JSON of the API query is possible.

In this part, a so-called "list" widget is to be implemented, which enables the user to receive only those issues that are in the selected status. The widget should look like Figure 1.

After this tutorial you can:

• implement a list-widget.
• tailor the output of an input component to user input.

1. Extension of the dataset

As already described in the previous parts of the tutorial, the dataset is responsible for providing user input. For a further functionality we have to extend the layout by the corresponding input field. The dataset class can be found at "src→main→java→de.odisys.talend.components→dataset→CustomDataset".

@GridLayout({
        @GridLayout.Row({"datastore"}),
        @GridLayout.Row({"projectId"}),
        @GridLayout.Row({"status"})
})

After the layout has been extended, an associated attribute must be defined in the class. Important for the "List"-Widget is the annotation "@Proposable("valuesProvider")". Under the name "valuesProvider", the widget will now search for the data it should display as selection values. This value provider is implemented in Chapter 2.

@Option
@Proposable("valuesProvider")
@Documentation("")
private String status;

All changes result in the following dataset:

@DataSet("CustomDataset")
@GridLayout({
        @GridLayout.Row({"datastore"}),
        @GridLayout.Row({"projectId"}),
        @GridLayout.Row({"status"})
})
@Documentation("")
public class CustomDataset implements Serializable {
    @Option
    @Documentation("")
    private CustomDatastore datastore;

    @Option
    @TextArea
    @Documentation("")
    private String projectId;

    @Option
    @Proposable("valuesProvider")
    @Documentation("")
    private String status;

    public CustomDataset(CustomDatastore datastore, String projectId, String status) {
        this.datastore = datastore;
        this.projectId = projectId;
        this.status = status;
    }

    public CustomDataset() {}
    public CustomDatastore getDatastore() { return datastore; }
    public String getProjectId() { return projectId; }
    public String getStatus() { return status; }
    public void setStatus(String status) { this.status = status; }
    public void setDatastore(CustomDatastore datastore) { this.datastore = datastore; }
    public void setProjectId(String projectId) { this.projectId = projectId; }
}

Don't forget: At least the display name ("_displayName") must be declared in the properties file under "src→main→ressources→de.odisys.talend.components→dataset→Messages.properties" for the new attribute!

CustomDataset.datastore._displayName=Datastore
CustomDataset.projectId._displayName=Project Id
CustomDataset.status._displayName=Status

2. Configuration in the Service File

After the input has been added in the dataset, the so-called "value provider", i.e. the provision of the possible values of the list widget, must be implemented. This is done in the "Service" file of the project. In case of our Jira component it is located at "src→main→java→de.odisys.talend.components→service→OdiSysService".

@Service
public class OdiSysService {
    @DynamicValues("valuesProvider")
    public Values vendors() {
        return new Values(asList(new Values.Item("1", "All"),
                new Values.Item("2", "Open"),
                new Values.Item("3", "In Progress"),
                new Values.Item("4", "Done"),
                new Values.Item("5", "To Do"),
                new Values.Item("6", "In Review"),
                new Values.Item("7", "Under Review"),
                new Values.Item("8", "Approved"),
                new Values.Item("9", "Cancelled"),
                new Values.Item("10", "Rejected"),
                new Values.Item("11", "Draft"),
                new Values.Item("12", "Published")));
    }
}

A method annotated with "@DynamicValues("valuesProvider")" needs to be implemented. "valuesProvider" represents the tag under which the widget searches for the method. A list of values that are available for selection is to be returned. We have chosen some possible states of a Jira issue.

3. Output Processing in the Source

The last step is the processing of the output data based on the entries made by the user. Here we implement a method "filterStatus()", which receives the JSON data from the Jira interface and returns a filtered selection. This selection only contains the issues that have the status which the user selected.

These requirements can be easily implemented with the library "JsonPath" (we have already integrated it into the project in Tutorial 1).

public String filterStatus(String resString) {
    if (configuration.getDataset().getStatus().equals("All")){
        return resString;
    } else {
        return JsonPath.read(resString, "$.issues[?(@.fields.status.statusCategory.name =~ /.*" 
                                           + configuration.getDataset().getStatus() 
                                           + "/i)]")
                                  .toString();
    }
}

Now the just implemented method must be applied to the data of the Jira-API. This results in the following source class:

@Documentation("")
public class OdiSysInputSource implements Serializable {
    private final OdiSysInputMapperConfiguration configuration;
    private final OdiSysService service;
    private final RecordBuilderFactory builderFactory;
    private boolean executed = false;

    public OdiSysInputSource(@Option("configuration") final OdiSysInputMapperConfiguration configuration,
                             final OdiSysService service,
                             final RecordBuilderFactory builderFactory) {
        this.configuration = configuration;
        this.service = service;
        this.builderFactory = builderFactory;
    }

    @Producer
    public Record next() {
        if (!executed) {
            String resString = "";
            try {
                HttpResponse<String> res = Unirest.get(
                        configuration.getDataset().getDatastore().getUrl()
                                + "/rest/api/2/search?jql=project="
                                + configuration.getDataset().getProjectId())
                        .basicAuth(configuration.getDataset().getDatastore().getUsername(),
                                configuration.getDataset().getDatastore().getPassword())
                        .asString();
                ;
                resString = res.getBody();
            } catch (UnirestException e) {
                e.printStackTrace();
            }
            executed = true;

            resString = filterStatus(resString);

            return builderFactory.newRecordBuilder().withString("data", resString).build();
        }
        return null;
    }

    public String filterStatus(String resString) {
        if (configuration.getDataset().getStatus().equals("All")){
            return return JsonPath.read(resString, "$.issues[*]").toString();
        } else {
            return JsonPath.read(resString, "$.issues[?(@.fields.status.statusCategory.name =~ /.*" 
                                               + configuration.getDataset().getStatus() 
                                               + "/i)]")
                                      .toString();
       }
   }
}