Social media includes web-based and mobile based technologies which are used to turn communication into interactive dialogue between organizations, communities, and individuals. Andreas Kaplan and Michael Haenlein define social media as "a group of Internet-based applications that build on the ideological and technological foundations of Web 2.0, and that allow the creation and exchange of user-generated content." Social media is ubiquitously accessible, and enabled by scalable communication techniques.
# tcpdump -vvv -A -i eth0 'port 389'
The -vvv option turns on very verbose (aka noisy) output - there is also -vv, -v and none at all to reduce the noise. The -A option displays the packet content in plain ASCII. The last parameter in quotes specifies to monitor only traffic on port 389 of network interface eth0, e.g. the LDAP traffic we are interested in. There are many more parameters to tweak the monitoring settings - consult the man pages for details.
Analyzing the output we learned that users who could authenticate where queried against LDAP, but the ones who could not where not. This helped pinpoint the issue: Authentication failed if a user is not listed in the TWikiUsers topic.
There are other networking monitoring tools available for specific needs, see the related links below.
As a closing comment, network monitoring tools allow one to sniff network traffic. With power comes responsibility. Use it wisely and respectfully.
Related: | %IMGLINK{ URL="http://renderman.escapestudios.com/renderman/bin/view/Main/WebHome" IMG="pixar-renderman" }% | RenderMan TWiki, Escape Studios, Pixar, Disney | %IMGLINK{ URL="http://www.futuremelbourne.com.au/" IMG="futuremelbourne" }% | Nice case study of a public TWiki. Look for the PDF in "Future Melbourne Wiki Post-Implementation Review" section. | |||
| %IMGLINK{ URL="http://loop.ph/bin/view/Loop/WebHome" IMG="loop-ph" }% | Loop.pH: Example of nicely skinned TWiki by a design studio in London | %IMGLINK{ URL="http://collaborate.nist.gov/twiki-cloud-computing/bin/view/CloudComputing/WebHome" IMG="nist-cloud" }% | NIST Cloud Computing Collaboration site, used to develop standards and guidelines for the government’s quick and secure adoption of cloud computing. | |||
| See more samples of public TWiki sites. | ||||||
I am writing to you as a voter in your district. I urge you to vote "no" on cloture for S. 968, the PROTECT IP Act, on Jan. 24th. The PROTECT IP Act is dangerous, ineffective, and short-sighted. It does not deserve floor consideration. I urge my representative to vote "no" on SOPA, the corresponding House bill. Over coming days you'll be hearing from the many businesses, advocacy organizations, and ordinary Americans who oppose this legislation because of the myriad ways in which it will stifle free speech and innovation. We hope you'll take our concerns to heart and oppose this legislation by voting "no" on cloture.As a result of the coordinated protest against these bills by thousands of websites today, USA Today reported that several senators backed down from supporting the bills. Success, at least for now! Thanks all for all the support you gave to keep the Internet open! ]]!> (last changed by PeterThoeny)
"Our TWiki has a very good track record of availability. It kept working as usual when one data center was lost. When another data center outage happened, TWiki stopped working because the network attached storage was lost despite the high availability design. But TWiki recovered faster than the other content management platforms. There are internal users who don't want to put their content on other content management platforms than TWiki because other platforms don't provide high enough availability."TWiki-5.1 Community Release Feature Highlights: New features in the patch release are marked as "added in TWiki-5.1.1".
New TWiki-5.1.1 release available: Enterprise Collaboration. Rock Solid. Usable. Easy. Fun: http://bit.ly/twRel511 #e20 #enterprise #wikiYour help in spreading the word of the new release is greatly appreciated! Let us know what you think of this new release! ]]!> (last changed by PeterThoeny)
RFC:4180:
%INCLUDE{
"http://tools.ietf.org/html/rfc4180"
pattern=".*?<span class=\"h1\">(.*?)</span>.*"
}%
Which results in the following output:
http://example.com/zip2city?zip=10001. Assuming the ZIP code is supplied via a URL parameter to the current TWiki page, it is possible to pass that along to the included page:
City for ZIP code %URLPARAM{ "zip" encode="moderate" }%:
%INCLUDE{
"http://example.com/zip2city?zip=%URLPARAM{ "zip" encode="url" }%"
pattern=".*?<h1>(.*?)</h1>.*"
}%
Which results in the following sample output:
| Registered users: Sep 2011 | TWiki topics at CERN Sep 2011 | ||
|---|---|---|---|
First name,Last name,Age Jimmy,Neutron,14 Mickey,Mouse,16Fields may contain commas, double-quotes, and line breaks (CRLF); in which case they should be enclosed in double-quotes. If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double-quote. Example:
Field A,Field B,Field C,Field D normal field,"field, with, commas","field with line feed","field with ""quoted"" text"Now that we understand the basics of CSV, let's see how we can generate a CSV file from TWiki content. As long as content is structured using TWiki forms it is easy to generate reports on the fly using a SEARCH variable. The search can be formatted to produce output in CSV format. Let's look at the ContactDbAddOn, which is a contact database. Each record is a TWiki page that has a ContactForm with fields Salutation, Name, Job Title, Company, Phone, Mobile, Fax, Email, and URL. Here is a search that shows all records using the default search format:
%SEARCH{
"form.name = 'ContactForm'"
type="query"
excludetopic="*Template"
nonoise="on"
}%
Now let's add a search format to generate the CSV output. We add a header line containing the names of the fields (the format parameter should be on one line) :
%SEARCH{
"form.name = 'ContactForm'"
type="query"
excludetopic="*Template"
nonoise="on"
header="Salutation,Name,Job Title,Company,Phone,Mobile,Fax,Email,URL"
format=""","","","",
"","","","","""
}%
This search already produces the proper CSV output with fields enclosed in double-quotes. There are some limitations though:
<noautolink>
%SEARCH{
...
}%
</noautolink>
3. Even if a page only contains just the SEARCH, the TWiki skin adds headers with pulldown menus and footers: We can suppress all output except the SEARCH by specifying the text skin and a content type of plain text. Assuming we have a ContactsAsCSV page in the Sandbox web that contains just the <noautolink> tag and the SEARCH, we get the plain text result by adding these URL parameters to the page URL:
http://example.com/do/view/Sandbox/ContactsAsCSV?skin=text;contenttype=text/plain4. Content is delivered as HTML, e.g. shown in the browser: The user expects to see a "save file" dialog box. We change the content type from plain text to
text/csv, and set the content-disposition to attachment; filename=Contacts.csv as defined in RFC 1806 to suggest the file name Contacts.csv in the save-as dialog box. Here is a link that points to the ContactsAsCSV page with proper parameters (the link should be all on one line) :
<a href="http://example.com/do/view/Sandbox/ContactsAsCSV?skin=text; contenttype=text/csv%0Acontent-disposition%3A+attachment%3B+filename%3DContacts.csv" target="_top">Export to CSV file</a>That's basically it. We can further improve this, such as generating a suggested file name with current date. You are invited to download and install the ContactDbAddOn, it has a working export to CSV feature. References:
|
David Spark interviewing Peter Thoeny: Two Adopted Real-World Solutions for Enterprise Collaboration. |
Collaboration across the enterprise is quite the bugaboo. Getting yourself and a friend to collaborate on a project is easy. Getting everyone in your group to collaborate is not so difficult either. But getting an entire company to collaborate using social tools such as wikis and blogs is quite a tantamount task. It's not easy and it takes a lot of trial and error and coaxing. At Silicon Valley Code Camp, I chatted with Peter Thoeny, CTO of Twiki, which started out as a wiki for the workplace. But it's also a box of LEGO, said Thoeny, you can use to build your own web or workflow application that can run within the wiki. Thoeny offered up a couple simple solutions for enterprise collaboration: Replace "Email All" with internal blogs -- One client of Twiki's was having a problem with employees taking advantage of the "Email All." The casual "send to all" for multiple messages a day was clogging up the company's mail servers. Twiki proposed a series of department-specific internal corporate blogs that allowed for editorial control. The content was very visible on the company's homepage, and a weekly email was sent out to all employee's offering highlights from the blog. The solution reduced the "Email All" messages to just one a day. Wikifying a company -- File attachments are so yesterday and they're also extremely limiting to only those people who receive them. And those people often don't save or store them in a place for them to find later. Conversely, if the document is made publicly available on a wiki, and you just send a link, now that content is stored in an open corporate setting and has value to all, not just the intended recipient.David interviewed many people at the Silicon Valley Code Camp. Here are two video compilations that are really funny:
| Annoying Habits of Developers. | Your Worst Manager Ever |
|
Want to surprise your friends on Facebook, Twitter or Google Plus?
Type text: ¡uʍop ǝpısdn ɹɐǝddɐ ןןıʍ ʇı
|
onkeyup Javascript hook in the first text box to call an upsideDownText() function each time a key is released:
We do the text processing in the<textarea rows="5" cols="70" id="src" onkeyup="upsideDownText()"></textarea>
upsideDownText() Javascript function:
<script type="text/javascript">
function upsideDownText() {
var srcText = document.getElementById( 'src' ).value.toLowerCase();
var out = '';
for( var i = srcText.length - 1; i >= 0; --i ) {
var ch = srcText.charAt( i );
if( ch == 'a' ) {
out += '\u0250' }
} else if( ch == 'b' ) {
out += 'q' }
} else if( ch == 'c' ) {
out += '\u0254'
// etc....
} else {
out += ch
}
}
document.getElementById( 'dest' ).value = out;
}
</script>
First we get the content of the text box identified by id="src". We only process lower case letters, so we convert the string to lowercase using the toLowerCase() method. We could try to convert upper and lowercase characters; that is an exercise left to the interested reader.
Then we loop through the string, character by character, starting from the end of the string. A big if-then-else-if block handles the character conversion. Finally we push the converted string into the text box identified by id="dest", which is the lower text box.
That's it, pretty simple code. View the raw TWiki page to see the complete HTML and Javascript.
]]!>
(last changed by PeterThoeny) "Our TWiki has a very good track record of availability. It kept working as usual when one data center was lost. When another data center outage happened, TWiki stopped working because the network attached storage was lost despite the high availability design. But TWiki recovered faster than the other content management platforms. There are internal users who don't want to put their content on other content management platforms than TWiki because other platforms don't provide high enough availability."Use Cases - "Mission Critical" Morgan Stanley uses the platform for sharing and collaboration of content, both documents and web content. Here are their specific uses:
"TWiki not being a trading system or settlement system, TWiki's failure doesn't make the firm lose money directly. But some of mission critical systems depend on TWiki for their operation. In day-to-day operation and switching to new releases, operation manuals' availability is crucial. A very good track-record of availability and global content replication make TWiki appealing."Competitive Value TWiki has enjoyed over a decade of providing content collaboration solutions to our user base with millions of users using the open source community edition. But the competitive advantages of Twiki sometimes need repeating. A customer testimonial is the most effective way to community this. Here is what Morgan Stanley views as competitive advantages of Twiki against other content management systems:
There is a way to use regular expressions to parse, say, up to 5 levels, and with the limitation that you can have only one function call per level. Certainly we need a more flexible solution that does not limit us on the number of levels for nesting and number of function calls per level. The solution is to parse twice, once to temporarily label the parentheses with level of nesting, then to resolve function calls recursively using matching parentheses identified by the labels. This can be done with regular expressions and recursive function calls. First, let's determine the level of nesting, indicated by the numbers above the parentheses:$ROUND( $TIME( ), $TIMEDIFF( $TIME( $T( R$ROW( ):C$COLUMN( -1 ) ) ), day ) )
1 2 2 2 3 4 5 5 5 5 4 3 2 1
$ROUND( $TIME( ), $TIMEDIFF( $TIME( $T( R$ROW( ):C$COLUMN( -1 ) ) ), day ) )
We use a regular expression to add the nesting level to the parentheses, together with an escape token (so that we can easily identify it later for processing and removal). Escape tokens with level are indicated by -esc-N; newlines are added for clarity:
First, we write the main function that parses and resolves a spreadsheet formula:$ROUND-esc-1( $TIME-esc-2( -esc-2), $TIMEDIFF-esc-2( $TIME-esc-3( $T-esc-4( R$ROW-esc-5( -esc-5):C$COLUMN-esc-5( -1 -esc-5) -esc-4) -esc-3), day -esc-2) -esc-1)
sub _doFormula
{
my( $formula ) = @_;
my $level = 0;
$formula =~ s/([\(\)])/_addNestingLevel($1, \$level)/geo;
$formula = _doFunc( "MAIN", $text );
return $formula;
}
This function does two things:
_addNestingLevel(). We use modifiers geo, e.g. g for global (don't stop at the first match), e for execute (to call a function for each match), and o to compile the regular expression pattern only once (because the pattern is fixed). _doFunc() to resolve the functions recursively.
_addNestingLevel() function adds the escape tokens with the level of nesting to a parenthesis, such as -esc-5( and -esc-5):
my $escToken = "\0";
sub _addNestingLevel
{
my( $theParen, $theLevelRef ) = @_;
my $result = "";
if( $theParen eq "(" ) {
# Increment level before adding escape token
$$theLevelRef++;
$result = "$escToken$$theLevelRef$theParen";
} else {
# Increment level after adding escape token
$result = "$escToken$$theLevelRef$theParen";
$$theLevelRef--;
}
return $result;
}
The nesting level is passed as a reference so that we can modify it. The escape token is the null character, which is a valid character in a Perl string. For the opening parenthesis "(", the nesting is incremented before adding the escape token with the nesting level. For the closing parenthesis ")", it is incremented after the fact. This ensures that we have the same nesting level for matching parentheses.
The _doFunc function handles all spreadsheet functions:
sub _doFunc
{
my( $theFunc, $theAttr ) = @_;
# Handle functions recursively
$theAttr =~ s/\$([A-Z]+)$escToken([0-9]+)\((.*?)$escToken\2\)/_doFunc($1,$3)/geo;
# Clean up unbalanced mess
$theAttr =~ s/$escToken\-*[0-9]+([\(\)])/$1/go;
my $result = "";
if( $theFunc eq "MAIN" ) {
$result = $theAttr;
} elsif( $theFunc eq "ROUND" ) {
$result = _handleROUND( $theAttr );
} elsif( $theFunc eq "TIME" ) {
$result = _handleTIME( $theAttr );
# etc.
}
return $result;
}
The first regular expression s/\$([A-Z]+)$escToken([0-9]+)\((.*?)$escToken\2\)/_doFunc($1,$3)/geo finds all functions of format $NAME-esc-N( ... -esc-N) and calls the _doFunc function recursively to resolve the spreadsheet function. How can we identify the matching closing parenthesis of a function? The ([0-9]+) after $escToken captures the level. With \((.*?)$escToken\2\) we scan non-greedily over the text to find the first escape token of same level, indicated with \2. We use the same geo modifiers for the regular expression, the g modifier results in parsing the string from left to right to find and handle all spreadsheet functions of the same level.
Spreadsheet functions at a deeper level are resolved first because of the recursion. The combined use of recursion and g modifier results in an inside out / left to right handling of the spreadsheet functions, which is the desired behavior.
The second regular expression s/$escToken\-*[0-9]+([\(\)])/$1/go cleans up escape tokens that are left over. For example $EVAL( (2+7)/3 ) has parentheses that are not part of a function construct. Or, the formula might have unbalanced parentheses.
Finally we have a long if-then-else-if-then switch that handles all spreadsheet functions.
That is basically it! As you can see, with a few lines of code it is possible to parse complex nested structures by using just a few regular expressions and a function recursively.
This is a simplified presentation of the actual implementation of the SpreadSheetPlugin. Some spreadsheet functions get special treatment. For example, the $IF( condition, then, else ) function needs to evaluate first the condition, then handle either the then or the else part based on the condition. This complicates the simple inside out / left to right parsing/handling described in this article. Interested folks are invited to download the SpreadSheetPlugin and study the code.
It would be interesting to do a reference implementation using the LL-parser/tree parser approach so that we could compare the performance to the pure regular expression/recursion approach discussed here. I have a hunch that the regular expression approach is an order of magnitude faster. Anyone challenging that? I would be happy to be proven wrong!
I hope this advanced regular expression topic is a learning experience for you as it certainly was for me. May be you can use this for one of your own projects?
References: