Natural Language Understanding (NLU) No Concepts Returned

Question

I am extracting Concepts using IBM Watson Natural Language Understanding (NLU) API. For most texts it is able to extract at least 1 or 2 Concepts, however in some simple cases it returns no Concept.

from watson_developer_cloud import NaturalLanguageUnderstandingV1
from watson_developer_cloud.natural_language_understanding_v1 import Features, CategoriesOptions, ConceptsOptions, RelationsOptions

natural_language_understanding = NaturalLanguageUnderstandingV1( version='2018-11-16', iam_apikey='API-KEY', url='https://gateway.watsonplatform.net/natural-language-understanding/api')

post ="No job Never had any romantic experiences I just have no ability / infrastructure to get through life It's killing me I don't want to be part of this world because I can't fit in, can't compete, can't enjoy Why does it have to be so uncomfortable? I feel so sad on the inside Another night I wonder how it will ever change, will it require my effort completely? I DON'T KNOW WHAT TO FUCKING DO"

response = natural_language_understanding.analyze(
    text=post, features=Features(
            concepts=ConceptsOptions(limit=10))).get_result()

The result returned in the response variable is

{'concepts': [], 'language': 'en', 'usage': {'features': 1, 'text_characters': 393, 'text_units': 1}}

Is it a known limitation of API that I am unaware of or is there some issue with the way I am calling the API?

Answer 1

For the same text I am able to get the following the as output for concepts from the API with same limits and same version date.

"concepts": [
    {
        "text": "Existentialism",
        "relevance": 0.988784,
        "dbpedia_resource": "http://dbpedia.org/resource/Existentialism"
    },
    {
        "text": "The Brothers Karamazov",
        "relevance": 0.856152,
        "dbpedia_resource": "http://dbpedia.org/resource/The_Brothers_Karamazov"
    },
    {
        "text": "Fyodor Dostoyevsky",
        "relevance": 0.83996,
        "dbpedia_resource": "http://dbpedia.org/resource/Fyodor_Dostoyevsky"
    },
    {
        "text": "Human condition",
        "relevance": 0.716108,
        "dbpedia_resource": "http://dbpedia.org/resource/Human_condition"
    },
    {
        "text": "Psychology",
        "relevance": 0.687781,
        "dbpedia_resource": "http://dbpedia.org/resource/Psychology"
    },
    {
        "text": "Philosophy of life",
        "relevance": 0.675628,
        "dbpedia_resource": "http://dbpedia.org/resource/Philosophy_of_life"
    },
    {
        "text": "Meaning of life",
        "relevance": 0.65926,
        "dbpedia_resource": "http://dbpedia.org/resource/Meaning_of_life"
    },
    {
        "text": "Human nature",
        "relevance": 0.652501,
        "dbpedia_resource": "http://dbpedia.org/resource/Human_nature"
    }
]

Can you pass in some wikipedia text for any famous place to verify if you get concepts returned at all in the output. Also make sure you do not set any default language while sending information to the API. NLU auto detects the language if the text is more than 100 characters in most cases. For example if you set the language to spanish (es) it might invoke the spanish concepts engine and you may get few to no results.