performance - How to tag texts automatically while remaining effectivness? -

-

Say I have a set of one million tags and a text that could possibly be parsed for these and possibly new tags Is required . The amount of tags here is an example to illustrate the problem of my thinking - a lot of ways to keep memory, etc. through a lot of loop in a linear way.

In some way, I can not think of any solutions, low footprint (and which is fast), I know that someone expects trade-off, but I think I have some concepts Seeing.

This is especially interesting for intelligent tagging ("Michael Jackson" = "Artist", etc.) Since the applicable tag can not be part of the text

In addition to popular blacklisting, caching of popular tags and huge SQL queries, what would be the most effective way to get into it?

P> (Enough funny, I have to tag myself this question :-))

Since I am limited in comment place, add me some ideas here:

  • I agree that the use of integer hash improves speed, good idea.
  • Hashes will not solve problems (Looping through each Hush / Tag, while checking word or word combination against tag)
  • To refine the problem : Say "Hello World" Lesson There are 3 possible tags in this lesson ("Hello", "World" and "Hello World"). The tag list may contain only "hello", but after parsing "world" or "hello world" can be added, which will mean that these tags will not apply to text.

Problems:

  • Recognizing the text of the book, repeat through all combinations (like "nine inch nails" but we assume that Combination limit is 4 words) It takes a long time to compare it with the tag in the database, even treating the use of integer hash as well.
  • The tag list is potentially long, so archiving tags may also slow down the running.
  • Tag updates will mean additional full text searches on texts - based on the length of the texts and their length and possibly the DB killer, and not efficient at all?
  • How to tag a "relevant" automatically? (Again, "Nine Inches Nails" comes to mind in an article about music - but "a new song released" will not be a good tag). Perhaps this is probably a question on himself.

Hash each word in the incoming text and use it to match the hash of those tags To do what you want to match. You can use a database to store and view hash values ​​so that you do not need to do this in memory.


Comments

Post a Comment

Popular posts from this blog

c++ - Linux and clipboard -

-
After selecting the copies of the buffer in Linux, after the text, we can click and paste the button between the mouse. I think there is a special buffer for this, I want to use it. Programming Language: C ++ Your Library: Qt Thanks. There is just one more correct answer than Paul Dixon, which answers your needs: / P > QClipboard * clipboard = QApplication :: clipboard (); CastString Selected Text = Clipboard-> Text (QClipboard :: Selection);
Read more

What is expire header and how to achive them in ASP.NET and PHP? -

-
I have examined the performance statistics of my website today. I have received a warning (or error may occur) that is given below Add Terminal Headers Here are 15 static components without a 15-futures end date * (no end) Http://www.example.com/video/css/global.css * (no time limit) http://www.example.com/video/js/global.js * (no expiration ) Http: //www.example.com/video/images/main-bg.png what it means and how to achieve it both in PHP and ASP.Net. I am on a shared hosting server, so please tell me some ways to use the code, because I can not make any modifications at the end of the server. If I expire the header, then there is no chance that if I make changes to the CSS, then the user will not be able to remove them immediately because CSS and other files have a fixed time limit (1 month, week ) Are cached for. Is Is there any harm to using PHP? $ time = time () + 3 * 24 * 60 * 60; // 3 day headings ('End:'. GMDAT ('D, D MYH: i: s \ g \ m \ t',
Read more

sql server - How can I determine which of my SQL 2005 statistics are unused? -

-
After a few years, one of my biggest databases has deposited 73 figures out of its largest tables. With Indexes, I can run many types of reports and queries, how often / heavily specific indexes are used. What is an equivalent for data? We are running SQL 2005. To make statistics automatically managed and honest, I would like to "update statistics", "create statistics" and There is no information about any management except "drop statistics" orders. In theory, UPDATE should handle the addition and removal of statistical data. In addition, I've never heard about the amount of storage or storage taking in large amounts, so I'm not sure whether there is any reason for the alarm or not. If you are seeing a potential problem, then I would suggest running an update before, if it does not clean it, it is leaving and then it appears to be safe operation (because it is the only query optimizer Is for). I will not leave "left" in the
Read more