| View previous topic :: View next topic |
| Author |
Message |
bdejong
Joined: 20 Oct 2004
Posts: 17
|
 Posted: Wed Oct 20, 2004 4:37 pm Post subject: Using WordNet to categorize audio files with text notes |
 |
|
important notes before reading this thread
- This thread was created from an (private) email conversation between me and Robert... Robert asked me if he could post my emails and I didn't see why.
- I'm mostly trying to understand how wordnet works, the different approaches, the different language implementations, and looking at what is needed to have "full wordnet 2.0 functionality" in PHP, with the databases in MySQL!
- The guys at mtg ( http://www.iua.upf.es | http://www.audioclas.org ) have been working on this type of thing for over 2 years and I'm trying to learn how it all works by trying to implement parts of it in my favorite web language
--------------------------------------------------------
Hello Robert,
For a research project I'm looking into using wordnet for classifying
textualy-described audio data. I'm looking into using PHP/MySQL as a
backend, but nowhere does there seem to be anything freely available.
I obviously downloaded and deployed your DB files, but ... nowhere on
the net is there a freely available project that actualy USES your
databases (admitted, using them is easier than using the flat files, but
there is still work 'to be done'). I'm guessing you guys DO have such a
system and I was wondering if we could perhaps have a look at it,
perhaps even use it (with proper pointers and mentions, of course).
For example, I'm looking for a PHP version of the word-morphing
techniques in wordnet ("spelt" => "spell", ...) using the morph
dictionaries. Do you have a DB version of those as well?
Or, alternatively, if you have any pointers for me that would be very
nice as well.
Kindest regards,
- Bram de Jong
_________________
http://www.iua.upf.es/mtg
http://www.icmc2005.org
http://www.smartelectronix.com
http://www.musicdsp.org
Last edited by bdejong on Thu Oct 21, 2004 7:05 am; edited 1 time in total |
|
| Back to top |
|
 |
roschler
Site Admin
Joined: 20 Oct 2004
Posts: 91
|
| Posted: Wed Oct 20, 2004 4:38 pm Post subject: |
|
|
Bram,
Interesting project. Unfortunately all our stuff is written in Delphi Object Pascal, not PHP, and we don't use it for word-morphing. Instead we use the glosses for statistical analyses. We put up the files only a few weeks ago which is why you can't find anyone using yet, even though we logged quite a few downloads.
I wish I could be of more help but that's the gist of it. If I bump into anyone using our files with PHP, I'll let you know.
Thanks,
Robert.
Last edited by roschler on Wed Oct 20, 2004 5:06 pm; edited 1 time in total |
|
| Back to top |
|
|
bdejong
Joined: 20 Oct 2004
Posts: 17
|
| Posted: Wed Oct 20, 2004 4:48 pm Post subject: |
|
|
| Quote: |
Bram,
Interesting project. Unfortunately all our stuff is written in
Delphi Object Pascal, not PHP, and we don't use it for word-morphing.
Instead we use the glosses for statistical analyses. We put up the
files only a few weeks ago which is why you can't find anyone using
yet, even though we logged quite a few downloads.
|
Aha! That makes perfect sense I guess...
I also guess I'll be looking into just writing what I need in C instead
of PHP, as it seems there aren't that many people using PHP with wordnet...
| Quote: |
I wish I could be of more help but that's the gist of it. If I bump
into anyone using our files with PHP, I'll let you know.
|
That would be really nice of you.
- bram
_________________
http://www.iua.upf.es/mtg
http://www.icmc2005.org
http://www.smartelectronix.com
http://www.musicdsp.org |
|
| Back to top |
|
|
roschler
Site Admin
Joined: 20 Oct 2004
Posts: 91
|
| Posted: Wed Oct 20, 2004 4:49 pm Post subject: |
|
|
Bram,
Remember, if you write it in C, and then you decide to put your app on a web server, you'll have a long haul in front of you porting it.
On the other hand, if you're going to be doing a lot of word crunching, then you may end up needing the speed of C.
Out of curiosity, why are you doing this research project and what kind of audio files are they (why do they need describing)?
Robert.
Last edited by roschler on Wed Oct 20, 2004 5:06 pm; edited 1 time in total |
|
| Back to top |
|
|
bdejong
Joined: 20 Oct 2004
Posts: 17
|
| Posted: Wed Oct 20, 2004 4:49 pm Post subject: |
|
|
| Quote: |
Bram,
Remember, if you write it in C, and then you decide to put your app on a web server, you'll have a long
haul in front of you porting it.
|
Not if the application is called from within PHP...
I might make an application that just does word-morphing and nothing
more. Then use the DB for looking up hyper and hyponyms and all the
other wordnet things I should still learn about.
| Quote: |
On the other hand, if you're going to be doing a lot of word crunching, then you may end up needing the
speed of C.
|
Not sure yet how much, but speed would be nice.
| Quote: |
Out of curiosity, why are you doing this research project and what kind of audio files are they (why do
they need describing)?
|
the project is for ICMC (http://www.icmc2005.org) and is, bascialy, my
day-job from now until somewhere in september. The idea is the creation
of a sample-exchange website where people can exchange open-source
samples to create music with.
Each sample should be described by the user and these descriptions
(keywords?) will be used to categorise and search the samples.
I.e. search for "water" and find sounds described by "river" for example.
-bram
_________________
http://www.iua.upf.es/mtg
http://www.icmc2005.org
http://www.smartelectronix.com
http://www.musicdsp.org |
|
| Back to top |
|
|
roschler
Site Admin
Joined: 20 Oct 2004
Posts: 91
|
| Posted: Wed Oct 20, 2004 4:50 pm Post subject: |
|
|
Bram,
Hmm, this gets more interesting. I am also a musician and the editor for Progressive Rock section at MusicMoz.
Then here are my suggestions (remember these are strictly my opinions):
- Don't use C. Sure you can call C from PHP, but you're going to want your base level search, before the fancy stuff, to be a reasonable full text index search. Remember if this takes off, you may end up with thousands of entries and just as many web users. If the users have to wait for a linear search, you'll be able to support only a handful of simultaneous users at a time. PHP and MySQL (MyISAM tables with fulltext
indices), should do the deed.
Better said, if you need C for speed, you are already doing something that will grind the server to a halt when you try to service a lot of users.
Here's the trick, how do you use the power of WordNet without creating something that will be impossible to put on a web server?
- You can't use WordNet during the *execution* of a search, that is when scanning the database, that's for sure (IMHO)
- So you have to use it when parsing the user's query *before* executing the search. In essence, use PHP/WordNet/MySQL to create a set of queries that you will execute in series, against the full-text database index.
For example, given a search phrase and assuming for example purposes it contains only nouns:
- ask the user to clarify the sense for each keyword by showing them the possible senses for each keyword
- for each word and each selected sense
+ pull out the two immediate hypernyms
+ pull out the two immediate hyponyms
+ pull out the most popular meronyms (things that the noun is a part of, e.g. hand -> fingers),
limit them to about two (arbitrary choice).
+ pull out the most popular holonyms (things that the noun is part of e.g. fingers -> hand), limit them to about two (arbitrary choice).
+ pull out the two most popular antonyms (e.g. "soft" -> "hard", in case a description for a sample has "not hard rock" in it), limit them to two (arbitrary choice). .
+ For each item, do a fast fulltext search using it, accumulating the results between items
+ Post-process the results and present them to the user in an intelligent fashion.
To clarify my usage of the term items, using the example above, there would be about 11 items (2 for each WordNet usage, and 1 for the original term), which would be run in series against the MySQL database. That is, 11 MySQL fast full-text searches.
PHP will be fast enough for this, easily, since all the hard work is done by MySQL (a C database server).
Just my thoughts.
Robert.
Last edited by roschler on Wed Oct 20, 2004 5:08 pm; edited 1 time in total |
|
| Back to top |
|
|
bdejong
Joined: 20 Oct 2004
Posts: 17
|
| Posted: Wed Oct 20, 2004 5:05 pm Post subject: |
|
|
what timezone are you in?
It's getting VERY late here... (03:43!)
Pardon me for any writing mistakes...
| Quote: |
Hmm, this gets more interesting. I am also a musician and the editor
for Progressive Rock section at MusicMoz.
|
Nice! Well, the university that's paying me is doing research on
automatic classification of songs... Might be interesting to you 
| Quote: | Then here are my suggestions (remember these are strictly my
opinions):
|
Wow, long email. Very interesting as well!!
See below for plenty more opinionated questions
| Quote: |
- Don't use C. Sure you can call C from PHP, but you're going to
want your base level search, before the fancy stuff, to be a
reasonable full text index search. Remember if this takes off, you
may end up with thousands of entries and just as many web users. If
the users have to wait for a linear search, you'll be able to support
only a handful of simultaneous users at a time. PHP and MySQL
(MyISAM tables with fulltext indices), should do the deed.
|
The C code would only be used to parse and extract words from the
descriptions of users (i.e. non-realtime). However... you made me think:
when searches appear you need to do the same type of mappings in real
time (word morphing, etc) before being able to find things via wordnet
(or the wordnet DB).
Then again, doing these mappings before reaching into the DB for
searches could be quite fast, even when calling compiled C code from
PHP. The problem is that I don't want to be bugged down by having to
translate all the C code to PHP, which would be a waste of time (better
waste that time then learning Perl or another language I could use that
DOES have the wordnet API translated)
| Quote: |
Better said, if you need C for speed, you are already doing something
that will grind the server to a halt when you try to service a lot of
users.
|
Yes, but I wasn't looking at C for speed, I was looking at C because
wanting to code the whole thing in PHP requires me to rewrite the
word-morphing (etc etc) in PHP. I'm not entirely sure only the
word-mapping needs to be rewritten though: do you have any idea about this?
Can you do all queries I will need directly on the Database, rather than
having to fall back onto "wordnet code"? I can see how hype / hyper /
... nyms logically follow from the DB, but does wordnet contain other
bits that I would need to consider to rewrite in PHP in the long end?
| Quote: |
Here's the trick, how do you use the power of WordNet without
creating something that will be impossible to put on a web server?
- You can't use WordNet during the *execution* of a search, that is
when scanning the database, that's for sure (IMHO)
|
What do you mean by saying "use wordnet"? Do you mean the actual wn.exe application? Or do you rather mean
using the power/system of wordnet? I think we need to clarify this before we go any further.
| Quote: |
- So you have to
use it when parsing the user's query *before* executing the search.
In essence, use PHP/WordNet/MySQL to create a set of queries that you
will execute in series, against the full-text database index.
|
No, I only wanted to use the wordnet executable when translating words
to their basic forms ("eaten" => "eat").
| Quote: |
- ask the user to clarify the sense for each keyword by showing them
the possible senses for each keyword - for each word and each
selected sense
|
in a search this could be removed: just show the user with searches for
all possible senses. Imho this would remove the burden from the user: he
doesn't have to bother with "learning" wordnet.
| Quote: |
+ pull out the two immediate hypernyms
+ pull out the two immediate hyponyms
+ pull out the most popular meronyms (things that the noun
is a part of, e.g. hand -> fingers), limit them to about two
(arbitrary choice).
+ pull out the most popular holonyms (things that
(the noun is part of e.g. fingers -> hand), limit them to about two
(arbitrary choice).
+ pull out the two most popular antonyms (e.g.
"soft" -> "hard", in case a description for a sample has "not hard
rock" in it), limit them to two (arbitrary choice). .
+ For each item, do a fast fulltext search using it, accumulating the
results between items
+ Post-process the results and present them to the user
in an intelligent fashion.
|
Yes, this is very near the ideas I had... Not sure why you'd need
fulltext searches if you store the keywords in advance in the DB. I.e.
sound X is described by synset/... ID's A, B, C.
Also, I think giving the user some SIMPLE feedback before you jump into
advanced mode would be interesting: "search for broader terms", "search
for more specific terms", "megasearch" <- do the whole damn fandango,
with bells whistles, etc...
| Quote: |
To clarify my usage of the term items, using the example above, there
would be about 11 items (2 for each WordNet usage, and 1 for the
original term), which would be run in series against the MySQL
database. That is, 11 MySQL fast full-text searches.
|
I guess one could easily use stored procedures in MySQL to make
everything easier...
| Quote: |
PHP will be fast enough for this, easily, since all the hard work is
done by MySQL (a C database server).
|
Actually: there's a system running at MTG (place where I work)that does
almost EXACTLY what you just described. However, there's problems with
this system:
+ it uses Perl for everything, I'm not sure I want to learn a
hyper-cryptic language to use the system they are using. Perl is a
hackers language over anything else... And I've seen their code. It's
hacky. :-S
+ it's overly complex. they present search-results to the user with
lot's of wordnet terms. The user doesn't need to know these details,
only that if he searches for water, there's an automatic search for
river, which will amaze him.
+ they don't use mysql. they use Berkeley DB versions of wordnet,
accessed by Perl, and they're storing offsets into these files as
ID's... etc etc, it's just too damned hacky.
The good thing is that their system actualy WORKS (i.e. I have a
real-world example which I can follow and I know the results will be
usable), and it's really interesting to see how it nicely "classifies"
everything like it should. Searches are really powerfully.
Have a look at their wordnet editor, for example:
http://www.audioclas.org/wneditor/
Your thoughts are MOST appreciated on this, by the way!
- Bram
_________________
http://www.iua.upf.es/mtg
http://www.icmc2005.org
http://www.smartelectronix.com
http://www.musicdsp.org |
|
| Back to top |
|
|
roschler
Site Admin
Joined: 20 Oct 2004
Posts: 91
|
| Posted: Wed Oct 20, 2004 5:06 pm Post subject: |
|
|
Bram,
I'd do these point by point but don't have time for that. Hope this clarification works for you.
| Quote: |
>> "Translating words to their basic forms"
|
You seem to indicate the synset ID as your basic form. If that is true, and taking into account your criticism of systems that display WordNet/morphological jargon to the user (I agree that this is a bad idea
BTW), how are you going to distill the user's query into synset ID's you can use for your database search?
If you intend to do a purely automatic method, that could be trickier than you think. The latest paper by WordNet's Christiane Fellbaum indicates the scope and depth of the problem. If you say someone else is doing it already, well then it's over my head.
A full-text search on synset ID's would be problematic. However, you will have a real problem then. If you have a user query that distills to the ID's "100002471 100004891 10004191", how can you match these against the database records without resorting to a linear non-indexed - slooow search? Unless there's a database record that contains exactly those synset ID's in that order, or contains those sub-set ID's in their exact order as a substring, you won't get a match. Maybe I am missing something here.
When I said "using WordNet" during execution, I was not talking about any executable's or the like. I simply meant trying to do any kind of WordNet analysis (hypernym, hyponym, etc). while scanning the database for matches. That approach, with any sizable database would be a CPU hog (IMHO). Instead do it up front before searching the database.
I don't know Perl. I intend to learn it at some point, but only to help out with coding at Open Site. I don't like the look of the language personally and PHP does everything I need for web programming. I am a veteran C/C++ programmer (and several other languages) and have been using PHP for about a year. For me, PHP is a breeze since it is pretty much C without the static typing issues and some very nice additions to make web programming easier.
There's a reason I went with the Prolog database clauses and converted them to MySQL instead of using the other WordNet source formats as a base. In fact I tried using the WordNet2Mysql files at first, which were more closely modeled after the C WordNet data files. I found the Prolog tables and inter-table relationships much more logically organized than the original C model. If you are a competent PHP programmer, using the MySQL files to do any WordNet operations is an extremely simple series of MySQL commands.
Last, when I said "ask the user to clarify the senses" I was absolutely not referring to have them become WordNet skilled. You can get people to clarify WordNet senses by showing them the example glosses for each sense for a word, and asking them which one best represents the way they are using the word. Behind the scenes you take their choice and resolve it to a Synset ID.
Hope this helps.
Robert. |
|
| Back to top |
|
|
bdejong
Joined: 20 Oct 2004
Posts: 17
|
| Posted: Wed Oct 20, 2004 5:10 pm Post subject: |
|
|
Hello Robert,
thanks for your time, ...
here's some more Q and A's
| Quote: |
You seem to indicate the synset ID as your basic form. If that is
true, and taking into account your criticism of systems that display
WordNet/morphological jargon to the user (I agree that this is a bad
idea BTW), how are you going to distill the user's query into synset
ID's you can use for your database search? If you intend to do a
purely automatic method, that could be trickier than you think. The
latest paper by WordNet's Christiane Fellbaum indicates the scope and
depth of the problem.
|
Hmm, that's a good point... However, I think it's possible to do
searches that make sense without having the user clarrify each of his
senses. All senses of "water" relate in some way to water, so it makes
sense to search for all of them.
I think there aren't that many words with senses that mean completely
different things ("spelt" => type of weat / form of the verb spell), but
most sample-jargon is quite set ("bass" "drum", ... etc) will have
well-defined meanings when looking at the descriptions people gave for
their samples.
| Quote: |
A full-text search on synset ID's would be problematic. However, you
will have a real problem then. If you have a user query that
distills to the ID's "100002471 100004891 10004191", how can you
match these against the database records without resorting to a
linear non-indexed - slooow search? Unless there's a database record
that contains exactly those synset ID's in that order, or contains
those sub-set ID's in their exact order as a substring, you won't get
a match. Maybe I am missing something here.
|
1. I think i only makes sense to search for combinations up to 2
adjacent words, and after filtering stop-words.
2. you should only return results for searches for terms that are
actualy in the description of samples:
I.e. synthesizer means "person who synthesizes" and "instrument".
However I don't believe you'll find any search results that refer to the
first sense, so shouldn't present anything to do with the sense "person
..." to the user.
| Quote: |
When I said "using WordNet" during execution, I was not talking about
any executable's or the like. I simply meant trying to do any kind
of WordNet analysis (hypernym, hyponym, etc). while scanning the
database for matches. That approach, with any sizable database would
be a CPU hog (IMHO). Instead do it up front before searching the
database.
|
Ah, now I get what you mean. For each description of a sample, store a
set of hypo/hyper/... etc then search in this set instead of searching
the whole DB.
| Quote: |
I don't know Perl. I intend to learn it at some point, but only to
help out with coding at Open Site. I don't like the look of the
language personally and PHP does everything I need for web
programming. I am a veteran C/C++ programmer (and several other
languages) and have been using PHP for about a year. For me, PHP is
a breeze since it is pretty much C without the static typing issues
and some very nice additions to make web programming easier.
|
Same for me, ... I might not a be a "veteran" but I've been coding in
C++ for about 7 years now, so I know my way around... I've written quite
a few audio effect plugins, so, C++ / PHP is the way to go
| Quote: |
There's a reason I went with the Prolog database clauses and
converted them to MySQL instead of using the other WordNet source
formats as a base. In fact I tried using the WordNet2Mysql files at
first, which were more closely modeled after the C WordNet data
files. I found the Prolog tables and inter-table relationships much
more logically organized than the original C model.
|
What's the exact difference between the database files you have and the
original C modeled files?
| Quote: |
If you are a
competent PHP programmer, using the MySQL files to do any WordNet
operations is an extremely simple series of MySQL commands.
|
OK, that's good to hear... I was thinking that maybe the C-version of
wordnet offered more functionality than just "database access" based on
simple command line things.
| Quote: |
Last, when I said "ask the user to clarify the senses" I was
absolutely not referring to have them become WordNet skilled. You
can get people to clarify WordNet senses by showing them the example
glosses for each sense for a word, and asking them which one best
represents the way they are using the word. Behind the scenes you
take their choice and resolve it to a Synset ID.
|
Yes, that's exactly what I thought. Even perhaps going further: samples
have a VERY specific jargon attached to them. Especialy if they are
samples in the classical sanse of the word ("bass drum", "smooth bass",
"sine sweep", etc), so for most of the part it makes sense to do some
matches automaticaly for the user. Of course any type of automation is
hard, but I think it's worth trying to keep statistics of what people
choose and after a while start filtering out things that don't make
sense with statistics.
- bram
_________________
http://www.iua.upf.es/mtg
http://www.icmc2005.org
http://www.smartelectronix.com
http://www.musicdsp.org |
|
| Back to top |
|
|
roschler
Site Admin
Joined: 20 Oct 2004
Posts: 91
|
| Posted: Wed Oct 20, 2004 5:11 pm Post subject: |
|
|
Bram,
| Quote: |
What's the exact difference between the database files you have and
the original C modeled files?
|
That would be very hard to answer because it's like trying to compare two different logic paradigms that are very different. It has to do with the way you bounce between files and look up or synthesize relationships between words based on the field formats of the various files. Unless you spend some time working with the two different format yourself, it's hard to quantify.
Robert. |
|
| Back to top |
|
|
bdejong
Joined: 20 Oct 2004
Posts: 17
|
| Posted: Wed Oct 20, 2004 5:11 pm Post subject: |
|
|
| Quote: |
That would be very hard to answer because it's like trying to compare
two different logic paradigms that are very different. It has to do
with the way you bounce between files and look up or synthesize
relationships between words based on the field formats of the various
files. Unless you spend some time working with the two different
format yourself, it's hard to quantify.
|
OK. Thanks for that. I.e. the difference is not in the content, but
rather in the representation...
By the way, have you guys translated the word-morphing to Delphi?
I'm asking because I've been looking at the C-code (morph.c) in wordnet
and I must say I'm amazed it works so well...  Having a cleaner
example to go from might be more interesting.
By the way, i'm going to convert the exception lists to MySQL. Would you
like to include them? (exceptions as in "caught" => "catch"). I'll be
translating the word-morphing to PHP (no matter if you guys have a
Delphi version ), and of course I shall share it as well.
cheers,
- bram
_________________
http://www.iua.upf.es/mtg
http://www.icmc2005.org
http://www.smartelectronix.com
http://www.musicdsp.org |
|
| Back to top |
|
|
roschler
Site Admin
Joined: 20 Oct 2004
Posts: 91
|
| Posted: Wed Oct 20, 2004 5:12 pm Post subject: |
|
|
Robert ---------------------------------------------------------------
| Quote: |
i'm going to convert the exception lists to MySQL
|
Bram,
Although I examined the C data files I have not looked at the code, so I am not familiar with the C code instructions. I'm using PHP 4.x which correct me if I am wrong does not have exceptions yet? In fact doesn't version 5.x beta 4 still not have C-style exception handling?
http://www.oracle.com/technology/pub/articles/hull_asp.html
BTW, I am (right now) setting up a forum for WordNet developers. With your permission I would like to move this email thread and all the previous messages we sent to each other, over to the forum and continue things over there. Would that be alright with you (showing your messages to me)?
Also, I'd like to get some more people involved as I think there are at least a few of us with questions.
I'll send you the link as soon as I am done setting it up.
Feel free to ask any PHP/MySQL/WordNet questions. I'll do my best to help.
Thanks,
Robert. |
|
| Back to top |
|
|
bdejong
Joined: 20 Oct 2004
Posts: 17
|
| Posted: Wed Oct 20, 2004 5:13 pm Post subject: |
|
|
| Quote: |
Although I examined the C data files I have not looked at the code,
so I am not familiar with the C code instructions. I'm using PHP 4.x
which correct me if I am wrong does not have exceptions yet? In fact
doesn't version 5.x beta 4 still not have C-style exception handling?
|
Hehehe, I mean "exceptions" as defined in the "morph" code from wordnet.
I.e. they call exceptions to regular plural forms, for example:
regular: car => cars
exception: goose geese
Not exceptions in the coding kind of way.
| Quote: |
BTW, I am (right now) setting up a forum for WordNet developers.
With your permission I would like to move this email thread and all
the previous messages we sent to each other, over to the forum and
continue things over there. Would that be alright with you (showing
your messages to me)?
|
Of course, go right ahead. I'd love to talk to some more people about
this, especially from a practical point of view!
- bram
_________________
http://www.iua.upf.es/mtg
http://www.icmc2005.org
http://www.smartelectronix.com
http://www.musicdsp.org |
|
| Back to top |
|
|
dbrian
Joined: 20 Oct 2004
Posts: 2
|
| Posted: Wed Oct 20, 2004 6:07 pm Post subject: similar projects |
|
|
Like the forum. There are a lot of duplicate projects going on around Wordnet, and not the types that get usually get announced to the WM users list.
I'm not involved in this one, but I know of a very similar project going on with the "Grupo de Tecnologia Musical" at a university in Barcelona (http://www.iua.upf.es/mtg/). They're using my Perl Lingua::Wordnet module (http://search.cpan.org/~dbrian/Lingua-Wordnet-0.74/), which is why I know about it. I don't have any more details. |
|
| Back to top |
|
|
dbrian
Joined: 20 Oct 2004
Posts: 2
|
|
| Back to top |
|
|
|
