Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Pretty high-level and broad (which I thought was good). Audience is clearly real-world decision makers as opposed to techies like the HN crowd. Some issues:

1) NLP performance characterized to be better than vision systems. I don't think that is true.

2) Some minor facts are not right. E.g. OpenAI did GPT-3, not Google

3) I expected the set of exemplars for technology to be better researched. Siri and Alexa are NOT recommendation engines. Similarly, lacked the best of class examples on many fronts. This was the biggest issue in the paper.



Help make the paper better. What are some better exemplars for technology?


Sure .. happy to help (and to be clear, I did find the paper to be insightful even as a senior researcher. e.g. I was familiar with OODA but the SPAA was neat! Don't take my comments as too negative :) )

For recommendation systems, the top three examples that come to mind are TikTok, Layer 6 (a Canadian company that TD Bank acquired a few years back) and Netflix.

You may want to add Nerfs to the paper. They are the hot new algorithm out there. I am a scientist at a Canadian research lab and my very smart colleagues tell me it is the next best thing.

Automated vision is far head of NLP IMHO. NLP had it's Imagenet moment only at the advent of BERT (which was cira 2018? or so). Also, Transformers, which BERT and its progeny rely on, are massively compute and data hungry. They are also slow to run on today's chips. In my opinion, reason is that language benchmarks aren't as clearcut as vision. For instance, NLP researchers use BLEU scores, which are a pretty blunt instrument. I'd say NLP is even further behind than speech processing (which is now mostly based on DL). A key person behind Siri is Adam Cheyer btw .. he did Siri and then Bixby. The way these NLP systems work is pretty simple conceptually .. they break the problem to two steps. Intent Identification and then Slot filling. You can use DL for both steps but don't have to. Key issue with NLP systems is they are extremely brittle (a ton of work to customize). Dialog is pretty weak today, and that is partly due to the challenge in training signal.

You say 5000 images per images of a class. I know that was a ballpark but this seemed misleading. There are at least 2 problems I see. First, you need "different" examples .. seeing the same examples (e.g. from different viewpoints) does not help. Second, it really matters what the set of classes your model is trying to discriminate against. E.g. to differentiate apples and bananas, I likely need far fewer than 5000 examples since they are so visually distinct. Imagenet was a seminal moment not just because of the number of examples per class but because of the humongous number of classes (10K+).

For RL and robotics, there have been some neat advances. I was skeptical about RL's utility in practice (due to reasons you point out .. simulations vs. real-world, and especially the issue of faster than real-time) but am seeing it more and more in practice. E.g. 5/6G applications exist.

You may want to add coverage of some important emerging topics: multi-modal (matching vision to text and vice-versa), sensor fusion, student-teacher.

Paper didn't talk about any work from MIT's Han lab or their startup OMNIML? They just launched at the TinyML summit this year and they are hot! Also, TVM tech (startup behind it is called OctoML) is pretty important for on-device AI.

Those were some initial thoughts. If this is useful, can add to it later.


As an ML professor, I agree with all of these comments, especially the bit about Nerfs. Take a look at Waymo's use of the technology: https://waymo.com/research/block-nerf/


Very helpful. I'll update the post tomorrow. thanks.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: