test_analyze_entities: salience outside expected range #2274

Activity

1. tseaver

   added

   natural-language

   flaky

   on Sep 8, 2016
2. tseaver commented on Sep 8, 2016

   

   tseaver

   on Sep 8, 2016

   ContributorAuthor

   More actions

   Similar error in test_anaylze_entities_from_blob:

   ======================================================================
   FAIL: test_analyze_entities_from_blob (language.TestLanguage)
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/home/travis/build/GoogleCloudPlatform/google-cloud-python/system_tests/language.py", line 117, in test_analyze_entities_from_blob
       self._check_analyze_entities_result(entities)
     File "/home/travis/build/GoogleCloudPlatform/google-cloud-python/system_tests/language.py", line 77, in _check_analyze_entities_result
       self.assertTrue(0.7 < entity1.salience < 0.8)
   AssertionError: False is not true
3. dhermes commented on Sep 8, 2016

   

   dhermes

   on Sep 8, 2016

   Contributor

   More actions

   Indeed. Le sigh.

   $ nosetests system_tests/language.py --pdb
   > /usr/lib/python2.7/unittest/case.py(422)assertTrue()
   -> raise self.failureException(msg)
   (Pdb) u
   > .../gcloud-python/system_tests/language.py(77)_check_analyze_entities_result()
   -> self.assertTrue(0.7 < entity1.salience < 0.8)
   (Pdb) entity1
   <google.cloud.language.entity.Entity object at 0x7f1fc87e07d0>
   (Pdb) entity1.salience
   0.81992269
   (Pdb) 
   
4. dhermes

   assigned

   jgeewax

   on Sep 8, 2016
5. dhermes commented on Sep 8, 2016

   

   dhermes

   on Sep 8, 2016

   Contributor

   More actions

   @jgeewax Can you weigh in here? The actual output of the API seems to be a moving target. Note that the input is a simple sentence that hasn't changed:

   Michelangelo Caravaggio, Italian painter, is known for 'The Calling of Saint Matthew'.
   
6. dhermes commented on Sep 9, 2016

   

   dhermes

   on Sep 9, 2016

   Contributor

   More actions

   In the meanwhile I'm going to let this set of tests go green, working on a PR now
7. dhermes

   added a commit that references this issue on Sep 9, 2016

   Widening the salience range for natural language system tests.

   ...

   This issue will close once commit 1b81d0e is merged into the 'master' branch.

   1b81d0e
8. dhermes

   closed this as completedin #2283on Sep 9, 2016
9. dhermes commented on Sep 9, 2016

   

   dhermes

   on Sep 9, 2016

   Contributor

   More actions

   @jgeewax Can you weigh in here?
10. dhermes

    reopened this on Sep 9, 2016
11. jgeewax commented on Sep 16, 2016

    

    jgeewax

    on Sep 16, 2016

    Contributor

    More actions

    I expect that the underlying model used here will evolve and "get better" over time. We should assert that there is a salience that is non-zero -- but how "salient" something is as determined by the algorithm can't be frozen.

    If we get back a zero, missing, etc, that means that the API contract is just "broken", but 0.75 today and 0.82 tomorrow doesn't mean our client is busted...
12. jgeewax

    removed their assignment

    on Sep 16, 2016
13. dhermes

    added a commit that references this issue on Sep 16, 2016

    Loosening salience assumptions in language system test.

    ...

    This issue will close once commit 0bc8de4 is merged into the 'master' branch.

    0bc8de4
14. dhermes

    closed this as completedin #2331on Sep 16, 2016
15. JustinBeckwith

    assigned

    dhermes

    on Feb 1, 2021
16. parthea

    added a commit that references this issue on Oct 21, 2023

    chore: move samples from python-docs-sample (#45)

    ...

    d369a34
17. parthea

    added a commit that references this issue on Mar 6, 2026

    bug: updates `__eq__` comparison on TableConstraint (#2274)

    ...

    27b1f9f