1.4.1 Das CIS und die Wittgenstein Advanced Search Tools

Die Wittgenstein Advanced Search Tools (WAST) sind eine Kollektion von verschiedenen Anwendungen des CIS, die spezifisch auf das Korpus des Archivs in Bergen, Norwegen (WAB) entwickelt wurde.

WAST ist entstanden aus einer zunehmend gewachsenen Kooperation mit dem des WAB Archivs in Bergen (vertreten durch Alois Pichler) und Max Hadersbeck (CIS).

Für die Zusammenarbeit spielen folgende Aspekte eine wesentliche Rolle:

• begrenztes Material
• sehr reichhaltig aufbereitetes, sehr sauberes (kontrolliertes Material)

Wittgenstein wiederum ist ein Subteil des ehemaligen(?) europäischen Projekts unter der Leitung von Paolo D’Iorio (darunter, u.a. )

WAST Projektstruktur

1.4.2 WAB – Wittgenstein Archive Bergen

1.5.1 Verwalten von Komponenten in WAST

Komponenten-Maintainer können ihre Komponente in die Gruppe cis(Gitlab cis group 2014) überführen lassen – dazu ist dann eine Mail an Thomas notwendig (glaube ich). Vorteile daraus:

1. das Projekt wird im gitlab nicht mehr gegen das persönliche Projektlimit bei der RBG gezählt

2. das Projekt ist weniger “personenbezogen” und kann besser von Nachfolgern “geerbt” werden

Komponenten-Maintainer, die ihre Komponente in die WAST-Infrastruktur hinzufügen wollen, machen folgendes:

1. Zunächst, die Komponente als git-submodule[link: http://git-scm.com/book/en/Git-Tools-Submodules] adden:

git submodule add git@gitlab.cip.ifi.lmu.de:cis/wf.git components/wf

2. Am besten direkt die Remote-Bezeichnung[link: http://git-scm.com/book/en/Git-Basics-Working-with-Remotes] origin in upstream umbenennen (der Einheitlichkeit halber)

   git remote rename origin upstream

3. Im eigenen Projekt die wesentlichen Dateien anlegen (oder die, die man für nötig hält):

   • wast.build

   • wast.check

   • wast.start

   • wast.stop

   • wast.restart

   Beispiele hierzu finden sich hier:

   https://gitlab.cip.ifi.lmu.de/cis/wast-feedback/tree/master

2.7.1 Resource Text: TEI P5 conform XML

The XML-Transcription of Wittgenstein’s Nachlass Pichler (2010), which was developed at WAB in Bergen, annotates the texts in greatest detail. All deletions and substitutions etc. of Wittgenstein are annotated in the XML-texts. The latter is too detailed in order to be used by WAST and thus a new TEI P5 TEI Consortium (2009) compatible and reduced XML-Format (CISWAB) was defined. This text format is an optimal base for the cooperation between the Wittgenstein researchers and programmers in the field of computational linguistic. The CISWAB texts are extracted via XSLT-transformation from WAB’s XML-transcriptions of Wittgenstein’s Nachlass Pichler et al. (2009).

2.7.2 Resource Lexicon: Electronic full-form Lexicon

Successful text searches crucially rely on the availability of an electronic full-form lexicon. For the work on Wittgenstein’s Nachlass, we used CISLEX Guenthner and Maier (1994), one of the biggest electronic German full-form lexica, which has been developed at CIS over the last 20 years. We extracted all words from the texts available on Wittgenstein Source, the word information from CISLEX and extended it to the special lexicon, called WiTTLex. Each word-entry in WiTTLex is formatted according to the DELA format, defined at the Laboratoire d’Automatique Documentaire et Linguistique (LADL), Paris Gaston (1991). The lexicon entries contain the word’s full-form, lemma, lexicographical word form, inflection and semantic information. Search queries to WiTTFind can be grammatically processed with the help of WiTTLex. Figure [fig:dic] shows a short excerpt from the lexicon.

[H]

Advokat, Advokat.N+HUM:neM
gesagt, sagen.V:OZ
gesamte, gesamt.ADJ+NUM:aeFxp:aeFyp:aeFzp:aeNyp:\
 amUxp:neFxp:neFyp:neFzp:neMyp:neNyp:nmUxp
dagestanden, dastehen.V+#2
glänzende, glänzend.ADJ+ER+COL+Glanz
Fermat, .EN
Fermatsche, Fermat'sch.ADJ+EN
rot, rot.ADJ+COL+Grundfarbe
rötlicher, rötlich.ADJ+COL+Zwischenfarbe+KOMP:\
 deFxp:geFxp:gmUxp:neMxp:neMzp

[fig:dic]

2.7.3 The finder Application suite ``

In order to provide extended linguistic search capabilities for explorations from non-technical and non-linguistic backgrounds, a suite of tools was developed at CIS Seebauer (2012; Krey 2013; Fink 2013; Volos 2013). With a mixture of implicit and explicit conversions we are able to provide advanced linguistic search capabilities to researchers from different fields of the humanities. The next section provides a bottom up overview on the program suite ``, one section out of WAST, that is used to find text, i.e. to find utterances of Wittgenstein.

Every query to `` is internally transformed to a local-grammar-search-graph of the search terms Maurice (1997). In its simplest form these graphs are a chain of the search terms. But in general search-graphs can contain an arbitrary number of loops and branches to express complex search patterns Fink (2013). The system uses various strategies to match tokens in the text. It is able to use i.a. string-based matching, regular expressions, semantic and morphological information from dictionaries and arbitrary annotations from the searched text itself.

In order to provide a simple interface to the user and hide much of the complexities involved, queries from the user are automatically converted to local-grammar-search-graphs. A simple query language enables the user to search for sequences of arbitrary length or combine tokens with boolean operators Fink (2013). Furthermore, the system applies implicit conversions on the queries to generate more natural search results.

2.7.4 Implementation aspects of

The tool is implemented in a client-server architecture. There are three main tools involved in the processing of queries: A server-, a client- and a displaytool. The client merely takes queries from the user and applies some very basic transformation to them before sending the actual query to the server. These transformations mostly convert convenient wildcard expressions to valid more complex regular expressions.

On the server side each query is parsed and then transformed to a local-grammar, that is applied asynchronously to the different documents the query wishes to search. One running server instance is able to provide parallel searches over a various set of different documents. All results are then send back to the client.

The third tool is a simple displaying tool, that is able to output the result of the queries in different formats². Together these three tools provide the backend to present query results, that are further used by the WiTTFind front end to present the final results of queries.

[H]

[tab:wfbenchmark]

Table [tab:wfbenchmark] lists a view benchmarking results of different query types to a server instance running on a modern desktop PC³. The numbers each show the median of 10 different queries to the text of the Big Typescript All queries where limited to a maximum of 100 hits⁴. It should be noted, that the last two lines show the results of complex queries, that generate very few hits. These queries take a lot more time to finish (more than twice as long), since the whole document has to be processed and local grammars are used to disambiguate.

2.7.5 Lemmatized and inverted lemmatized word-search

The electronic full-form lexicon WiTTLex allows to perform lemmatized queries to the Nachlass. For example, the search for the word denken (think) returns all sentences which contain morphological variants of the word, like dachte, gedacht. We also implemented with the help of WiTTLex an inverted lemmatized search where we used the lemma of the queried word to produce all morphological variants of it. The word dachte leads to the lemma denken again, from which all word variants are derived.

2.7.6 Word-Form Search and Part-of-Speech Tagging

In the full-form lexicon, WiTTLex, every word is stored together with its lexical word form as it is defined in the German CISLEX (see Table [tab:tags]).

[H]

[tab:tags]

The finder allows for the use of word forms as placeholders for words in a query. For example, the query Die <ADJ> Farbe finds all sentences that contain the nominative feminine definite article Die, followed by an adjective, which precedes the noun Farbe (colour) as in sentence ([ex:farbe]).

[ex:farbe]

[ex:farbe:q] Die <ADJ> Farbe [ex:farbe:a] Ich könnte Dir die genaue Farbe der Tapete zeigen, wenn hier etwas wäre was diese Farbe hat.⁵

To further reduce the syntactic ambiguity of the word forms in the text, we additionally tagged the data with the Part-of-Speech (POS) treetagger Schmid (1994). The finder permits POS-tags to be used as placeholders for the selection of syntactic word forms, as defined in the Stuttgart-Tübingen-Tagset Schiller, Thielen, and Teufel (1999). The user can decide to search for a lexical word form (e.g. <ADJ>) or the syntactic word-form within the sentence (e.g. [ADJ*]).

2.7.7 Semantic Lexical Classification

In WiTTLex we classified nouns Strutynska (2012) and adjectives Krey (2013) semantically. According to the work by Langer and Schnorbusch Langer and (Hrsg.) (2005) we defined eleven classes for nouns (see: Table [tab:semnoun]) and eleven classes for adjectives (see: Table [tab:semadj]).

[H]

[tab:semnoun]

In our investigations in Wittgenstein’s Big Typescript (BT)⁶, we found that there are about 1800 nouns out of all 46000 words in the data (see: Table [tab:semnoun] and [tab:semadj]). All the Tags from the Tables can be used in WiTTFind. For example, the query <EN> und <EN> returns the sentence ([ex:frege]) in which the two proper names Fregeschen and Russellschen are joined by the coordinating conjunction and.

[ex:frege]

[ex:frege:q] <EN> und <EN> [ex:frege:a]Unzulänglichkeit der Fregeschen und Russellschen Allgemeinheitsbezeichnung.⁷

In cooperation with the Faculty of Philosophy, Philosophy of Science and the Study of Religion, at the University of Munich (Rothhaupt Rothhaupt (1996)), we implemented a first semantic classification of Wittgenstein’s colour vocabulary together with a special HTML-interface for querying colours in the Nachlass. We found, that five different categories for colours are optimal for Wittgenstein’s Nachlass: Grundfarbe (primary colour), Zwischenfarbe (intermediate colour), Transparenz (transparency), Glanz (gloss) and Farbigkeit (colourness) Krey (2013). Table [tab:sem] shows the different labels for colours and the number of their occurrences in the text.

In the web-frontend WiTTFind, the user can select between different colour categories (see: Table [tab:sem]), view statistics and query Wittgenstein’s Nachlass.

2.7.8 Sentence structure and Wildcards

For the Wittgenstein researchers it is very important to take the sentence structure into account. To enable users to specify the sentence structure within queries, we introduced sentence structuring tags in queries, such as <BOS> (the beginning of a senctence), <EOS> (the end of a sentence) and <PUNCT> for punctuation characters. The wildcard operator * can be used as placeholder for arbitrary character sequences. The query: <BOS> Ich meine nur <PUNCT> * * * <PUNCT> <EOS> would return all sentences that consist of six words starting with the sequence of three tokens Ich meine nur followed by a punctuation character as in example ([ex:sage])

[ex:sage]

[ex:sage:q] <BOS> Ich meine nur <PUNCT> * * * <PUNCT> <EOS> [ex:sage:a] Ich meine nur, was ich sage.⁸

[tab:sem]

|c|c|p2.0cm|c| Name & Tag & Translation & Occurrences
Grundfarbe & <Grundfarbe> & basic colour & 454
Zwischenfarbe & <Zwischenfarbe> & intermediate colour & 301
Transparenz & <Transparenz> & transparency & 105
Glanz & <Glanz> & gloss & 2
Farbigkeit & <Farbigkeit> & colourness & 29

2.7.9 Rule-based linguistic search with Part-of-Speech Tagging (POS-tagging)

To enrich the number of found verbs concerning a search query, we implemented an automatic particle-verbs detection and distinction. Particle-verbs are marked in their lexical entry and divided into verb and particle. In the Big Typescript we found almost 750 verbs with separated particles. To disambiguate the separated particles from prepositions, we use Part-of-Speech tagging and local grammars. For example, the query with the particle verb dastehen would extract instances as: steht klar da … in which the verb particle da, which may occur in German separately from the verb stehen, is recognized as such and not as a preposition (see: Figure [fig:dastehen]).

2.7.10 Search without Alternatives

One characteristic feature of Wittgenstein’s Nachlass is, that Wittgenstein changed his texts very often and as a result the Nachlass offers a lot of different readings. To enable the finding of remarks in all of the latter, in a background process of the finder application, we generate all different readings, which are processed simultaneously Seebauer (2012).

2.7.11 Search Result Layout with the Facsimile

Ludwig Wittgenstein’s Nachlass is highly heterogeneous. It consists of a large number of texts, some of them handwritten, with numerous passages edited from the author himself. The researchers can only appreciate the found remarks to the fullest with all its editions, errors and overall form, if they see not only the HTML-transformation of the edition, but as well their original facsimile which should be displayed simultaneously Rothhaupt (2006). Only this combined view provides the possibility for comparison and analysis of the original material, which carries the complete set of information. In order to implement this layout, it was necessary to OCR all facsimile of Wittgenstein’s Nachlass, extract the coordinates of his remarks and link them to the search result according to their siglum Gotscharek et al. (2011). All the work was done with ABBYY FineReader and additionally developed PERL programs. The highlighting in the display of the facsimile is done with CSS techniques within the HTML Page Lindinger (2013).

3.5.1 Projektstruktur

Hier ein Überblick und eine Beschreibung der wichtigsten Teile der Projektstruktur:

.
|-- build                   # Out-of-source dir für cmake / Kompilierung
|   |-- bin                 # Platz für die gebildeten exeucatables
|   |-- lib                 #            --- " ---     libraries
|   `-- src                 #            --- " ---     *.o
|-- cmake                   # cmake-module für Makefile-Generation
|-- etc                     # Test-Daten und anderes
|-- src                     # C++-Sources
|   |-- adapter             # Alle Adapter-Klassen (wrappen vendor/cautomata)
|   |   |-- VoidSequence
|   |   `-- inenagaCDAWG
|   |-- features            # Tests
|   |   `-- data            # Test-Daten
|   `-- indexer             # Indexer-Klassen
|-- vendor                  # 3rdparty libs (eingebunden als git submodules)
|   |-- cautomata           # C-Schicht von Petar Mitankin, Sofia
|   |-- make_unique         # wurde im C++11-Standard schlicht vergessen
|   |-- pugixml             # XML-Modul
|   |-- serialization       # ...
|   `-- testing
`-- web                     # Web-Schicht: MEAN-Stack
    |-- bower_components    # Client-seitige Javascript-Bibliotheken
    |-- build               # hier wird das node-addon gebildet
    |-- cpp                 # Hier liegen die Klassen, die src/ nach JS wrappen
    |-- node_modules        # Server-Seitige Module
    |-- public              # Client-Seite
    |   |-- javascripts
    |   `-- stylesheets
    |-- routes              # Server-Seite: Routes
    `-- views               # Client-Seite: Views
        `-- partials

3.5.2 Schichten

• Schicht 1: C⁹

  • Bereitstellung von SCDAWG-Automaten für einzelne Wörter/Wörterbücher

• Schicht 2: C++(11)¹⁰

  • Erweiterung der SCDAWGs aus der C-Schicht, um diese für die Suche innerhalb von Dokumenten (statt Wörtern) verwendbar zu machen

• Schicht 3: Web¹¹ mit:

  • Serverseite

    • Nodejs+ExpressJS

    • node addon als Wrapper der C++-Automaten (ermöglicht das Einbinden von C++-Objekten in Javascript code)¹²

  • Client-Seite

    • AngularJS¹³

SIS Schichten

3.5.3 Klassen

Zunächst ein Überblick:

Klassenarchitektur von SIS

Im Folgenden eine Kurz-Besprechung der (wichtigsten) Klassen, ihres Zwecks und ihrer grundlegenden Eigenschaften aus der C++-Schicht, welche die structs aus der C-Schicht wrappen und dokumentindexierende Eigenschaften zur Verfügung stellen:

<s n="Ms-115,3[2]_1" ana="facs:Ms-115,3 abnr:7 satznr:22">
  “Was heißt es: ‘dieser Gegenstand ist mir<lb/> wohlbekannt?’”
</s>

typedef Document<StringType>            key_type;
typedef DocumentPos<StringType>         value_type;
typedef std::map<key_type, value_type>  docinfo_t;
/// @todo rename: value_type -> mapped_type (do this everywhere)

3.5.3.5 DocumentIndexingAutomaton

Der DocumentIndexingAutomaton (Abbildung auf Seite ist die komplexeste Klasse hinsichtlich seiner Member. Er ist die zentrale Stelle, an der alle Informationen zusammenlaufen:

• Hinzufügen von Documents zu den C-Automaten und entsprechendes
• Indexing dieser Dokumente, d.h.
  • verwalten der sinkstates
  • und der Documents
• Retrieval und Suche auf den Automaten, und dabei
  • “Ausschneiden” von passenden Ergebnissen
  • und Auslieferung derselben an die Clients

DocumentIndexingAutomaton UML

3.5.3.6 DocumentIndexingAutomatonAbstract

Die Basis-Klasse für DocumentIndexingAutomaton (Abbildung auf Seite )

DocumentIndexingAutomatonAbstract UML

3.6.1 C++-Ebene

• TDD: Test Driven Development [link: http://www.agiledata.org/essays/tdd.html] bzw. [link: späteres tdd kapitel]
  • lest (Martin Moene 2013)
• make_unique (C++1y Standard Committee 2014)
• Git Submodules (“Git - Submodules” 2014): git submodules in cmake (Daniel Bruder 2014)
• andere vendor-submodules aufzählen
• Template Meta-Programmierung¹⁴
  • [TMP vorstellen]
• Serialization (siehe mehr zum Thema Serialisierung allgemein im Kapitel Serialisierung und wie Serialisierung in SIS gemacht wird im Kapitel Zweistufige Serialisierung)
  • cereal (uscilab 2014)
  • [zwei Serialization Module vorstellen]
  • [link: cereal]
  • [link: boost serialization]

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#ifndef _HEADER_FILE_COMPRESSED_AUTOMATON_
#define _HEADER_FILE_COMPRESSED_AUTOMATON_

#define mCompressedAutomatonGetStatesStored( aut ) ( (aut)->statesTransitions->seqStored )

#define mCompressedAutomatonGetTransitionsStored( aut ) ( (aut)->transitionsStart->seqStored )

typedef struct tCompressedAutomaton{
    UINT initialState;
    UINTSequence * statesTransitions;
    UINTSequence * statesNumberOfTransitions;
    UINTSequence * statesEnd;
    UINTSequence * transitionsStart;
    UINTSequence * transitionsTo;
    VoidSequence * data;
    TarjanTable * tt;
} CompressedAutomaton;

extern CompressedAutomaton * CompressedAutomatonInit( UINT symbolSize );
extern void CompressedAutomatonAddState( CompressedAutomaton * aut, UINT state );
extern void CompressedAutomatonAddTransition( CompressedAutomaton * aut, UINT stateFrom, UINT start, UINT stateTo );

// ...

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

/// @brief Adapter for CompressedAutomaton
template<class StringType>
struct CompressedAutomatonAdapter
  : virtual public CompressedAutomatonAbstract<StringType>
{

    using CompressedAutomatonAbstract<StringType>::traits::string_type;
    using CompressedAutomatonAbstract<StringType>::traits::symbol_size;

protected:
    using adaptee_handle = 
        std::unique_ptr< CompressedAutomaton, decltype(&::CompressedAutomatonFree)>;
    adaptee_handle adaptee_;

public:
    explicit CompressedAutomatonAdapter()
    : adaptee_ { ::CompressedAutomatonInit(symbol_size), &::CompressedAutomatonFree }{}
    virtual ~CompressedAutomatonAdapter() {}


//——————————————————————————————————————————————————————————————————————————//
//     M A P P I N G   I N T E R F A C E              (AutomatonAbstract)   //
//——————————————————————————————————————————————————————————————————————————//
public:
    virtual void shrink() override;
    virtual void write(FILE * fp) const override;

// ....

//——————————————————————————————————————————————————————————————————————————//
//     M A P P I N G   I N T E R F A C E            (CompressedAutomaton)   //
//——————————————————————————————————————————————————————————————————————————//
public:
    virtual void add_state(UINT state) override;
    virtual void add_transition(UINT stateFrom, UINT start, UINT stateTo) override;

1
2
3
4
5
6
7
8
9
10
11
12
13

// typical.h
typedef unsigned char U8;
typedef unsigned short int U16;
typedef unsigned int U32;
typedef unsigned long long U64;

typedef char S8;
typedef short int S16;
typedef int S32;
typedef long long S64;

typedef U32 UINT;
typedef S32 SINT;

1
2
3

// voidSequence.h
#define ENCODING_PLAIN (0)
#define ENCODING_UTF8 (1)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// compressedAutomaton.c
CompressedAutomaton * CompressedAutomatonInit( UINT symbolSize ){
    CompressedAutomaton * aut;

    aut = Malloc( 1, sizeof(CompressedAutomaton) );
    aut->initialState = NO;
    aut->statesTransitions = UINTSequenceInit2( 1024, 1024 );
    aut->statesNumberOfTransitions = UINTSequenceInit2( 1024, 1024 );
    aut->statesEnd = UINTSequenceInit2( 1024, 1024 );
    aut->transitionsStart = UINTSequenceInit2( 1024, 1024 );
    aut->transitionsTo = UINTSequenceInit2( 1024, 1024 );
    aut->data = VoidSequenceInit2( symbolSize, 1024, 1024 );
    aut->tt = NULL;
    return aut;
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

/// @brief Adapter for CompressedAutomaton
template<class StringType>
struct CompressedAutomatonAdapter
  : virtual public CompressedAutomatonAbstract<StringType>
{

    using CompressedAutomatonAbstract<StringType>::traits::string_type;
    using CompressedAutomatonAbstract<StringType>::traits::symbol_size;

protected:
    using adaptee_handle = 
        std::unique_ptr< CompressedAutomaton, decltype(&::CompressedAutomatonFree)>;
    adaptee_handle adaptee_;

public:
    explicit CompressedAutomatonAdapter()
    : adaptee_ { ::CompressedAutomatonInit(symbol_size), &::CompressedAutomatonFree }{}
    virtual ~CompressedAutomatonAdapter() {}

1
2
3
4
5
6
7
8
9
10
11

/// @brief interface for CompressedAutomaton
template<class StringType>
struct CompressedAutomatonAbstract
  : virtual public AutomatonAbstract<StringType>
{

    using AutomatonAbstract<StringType>::traits::string_type;
    using AutomatonAbstract<StringType>::traits::symbol_size;  

    virtual UINT get_symbol_size() const = 0;
    virtual CompressedAutomaton * read( FILE * fp ) = 0;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

namespace sis {

template <class StringType>
struct string_traits {
public:
    typedef typename ElementSize<StringType>::type  string_type;
    static constexpr unsigned                       symbol_size = ElementSize<string_type>::value;
};

/// @brief  Defines an Automaton in general
/// @todo   rename: AutomatonAbstract -> AdaptedAutomaton (?)
template<class StringType>
struct AutomatonAbstract : boost::noncopyable, string_traits<StringType> {

    using string_type = typename            string_traits<StringType>::string_type;
    static constexpr unsigned symbol_size = string_traits<StringType>::symbol_size;

//——————————————————————————————————————————————————————————————————————————//
//     C T O R  /  D T O R                                                  //
//——————————————————————————————————————————————————————————————————————————//
public:
    AutomatonAbstract() { mSetEndianness() } // this is important.
    virtual ~AutomatonAbstract() {}


//——————————————————————————————————————————————————————————————————————————//
//     M A P P I N G   I N T E R F A C E                                    //
//——————————————————————————————————————————————————————————————————————————//
public:
    virtual void shrink() = 0;
    virtual void write(FILE * fp) const = 0;

// ...

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

namespace sis {

/// @brief meta-class to handle StringType type traits (using static assertions)
template<class T>
struct ElementSize {
    static constexpr bool T_is_string_type =
                std::is_base_of< std::string, T>::value
             || std::is_base_of<std::wstring, T>::value ;

     static_assert(T_is_string_type,
         "\n\n\tYou must instantiate this class in question with std::string,"
         "\n\tstd::wstring, or any class derived thereof.\n"
     );


    typedef T                               type;
    typedef T                        string_type;
    typedef typename T::value_type    value_type;

    typedef typename std::conditional<
        std::is_same<std::string,T>::value, // if StringType is std::string
        std::fstream,                       // then stream_type is std::fstream
        std::wfstream                       // else it is std::wfstream
        >::type                             stream_type;

    static constexpr unsigned int value =
                            sizeof(typename string_type::value_type);
};

} /* sis */

#include "ElementSize.hpp"

template<class T>
class Foo {
  using stream_type = ElementSize<T>::stream_type;

  stream_type& operator<<(const T&);
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

#include <type_traits>
#include <cassert>

// C++11 Proof of concept of different implementations of
// member methods for different types of class instances.

template<class T> struct tag {};

template<class T>
struct Foo {

    // obviously, you can play with where you actually put these attributes
    // (at least with gcc)
    [[ dispatched_to ]]
    std::string bar(tag<std::string>) {
        return std::string{"foo"};
    }

    int bar(tag<int>) [[ dispatched_to ]] {
        return 5;
    }

    auto bar() [[ dispatcher ]]
    ->decltype(this->bar(tag<T>{})) // 'this' seems to be needed here.
    {
        return bar(tag<T>{});
    }
};

#ifndef DOCUMENTFIXTURE_HPP_8JMP5XLF
#define DOCUMENTFIXTURE_HPP_8JMP5XLF

#include "sis_config.hpp"

#include <vector>
#include <string>

typedef std::string file_t;                                                     /// @todo change this to fs::path

struct DocumentFixture
{
    std::vector<file_t> filenames = {
        sis_TEST_DATA_DIR"ascii/1.txt",
        sis_TEST_DATA_DIR"ascii/2.txt",
        sis_TEST_DATA_DIR"ascii/3.txt",
    };

    std::vector<file_t> wfilenames = {
        sis_TEST_DATA_DIR"utf8/1.txt",
        sis_TEST_DATA_DIR"utf8/2.txt",
        sis_TEST_DATA_DIR"utf8/3.txt",
    };

    std::vector<std::string> contents = {
        "Das Verstehen, die Meinung, faellt aus unsrer Betrachtung heraus.",
        "Also: Kann man Etwas anders, als als Satz verstehen?",
        "Oder aber: Ist es nicht erst ein Satz, wenn man es versteht."
    };

    std::vector<std::wstring> wcontents = {
        L"Das Verstehen, die Meinung, fällt aus unsrer Betrachtung heraus.",
        L"Also: Kann man Etwas anders, als als Satz verstehen? Ääääh...",
        L"Oder aber: Ist es nicht erßt ein Satz, wenn man es versteht."
    };
};

#endif /* end of include guard: DOCUMENTFIXTURE_HPP_8JMP5XLF */

#include "sis_config.hpp"

#include "testing/lest.hpp"
#include "DocumentFixture.hpp"

#include "indexer/Document.hpp"

#include <fstream>
#include <locale>
#include <cereal/types/vector.hpp>
#include <cereal/types/string.hpp>
#include <cereal/access.hpp> // For LoadAndAllocate
#include <cereal/types/memory.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/archives/json.hpp>
#include <cereal/archives/xml.hpp>

#include <cwchar>

DocumentFixture f;

const lest::test specification[] =
{
    "standard CTOR callable",[]
    {
        sis::Document<std::string> d;
        EXPECT( d.filename() == "" );
    },

    "CTOR with filename", []
    {
        sis::Document<std::string> d{ f.filenames.front() };
        EXPECT( d.filename() == f.filenames.front() );
    },

    "can serialize (automaton with added document)", []
    {
        sis::Document<std::string>  d{ f.filenames.front() };
        std::ofstream               ofs{ "document.bin" };
        cereal::BinaryOutputArchive oarchive{ ofs };

        oarchive( d ); // Write the data to the archive
    },

    "can deserialize (automaton with added document)", []
    {
        sis::Document<std::string>  d{ f.filenames.front() };
        std::ifstream               ifs{ "document.bin" };
        cereal::BinaryInputArchive  iarchive{ ifs };

        iarchive( d ); // Read the data
        EXPECT( d.filename() == f.filenames.front() );
        EXPECT( d.contents() == f.contents. front() );
    },

    "can serialize (automaton without added document)", []
    {
        sis::Document<std::string>  d;
        std::ofstream               ofs{ "document.bin" };
        cereal::BinaryOutputArchive oarchive{ ofs };

        oarchive( d ); // Write the data to the archive
    },

    "can deserialize (automaton without added document)", []
    {
        sis::Document<std::string>  d;
        std::ifstream               ifs{ "document.bin" };
        cereal::BinaryInputArchive  iarchive{ ifs };

        iarchive( d ); // Read the data
        EXPECT( d.filename() == "" );
        EXPECT( d.contents() == "" );
    },

    "can access content", []
    {
        sis::Document<std::string>  d{ f.filenames.front() };
        EXPECT( d.contents() == f.contents.front() );
    },

    "can access wide content", []
    {
        sis::Document<std::wstring>  d{ f.wfilenames.front() };

        /// Make sure to properly imbue!
        EXPECT( d.contents() == f.wcontents.front() );
    },

    "can access content (for default CTOR)", []
    {
        sis::Document<std::string>  d;
        EXPECT( d.contents() == "" );
    },

    "can access filename (for default CTOR)", []
    {
        sis::Document<std::string>  d;
        EXPECT( d.filename() == "" );
    },

    "can access filename (with given filename)", []
    {
        sis::Document<std::string>  d{ f.filenames.front() };
        EXPECT( d.filename() == f.filenames.front() );
    },

    "can serialize (automaton with added document) [as std::wstring]", []
    {
        sis::Document<std::wstring> d{ f.wfilenames.front() };
        std::ofstream               ofs{ "document.json.w" };
        cereal::JSONOutputArchive oarchive{ ofs };

        oarchive( d ); // Write the data to the archive
    },

    "can deserialize (automaton with added document) [as std::wstring]", []
    {
        sis::Document<std::wstring> d{ f.wfilenames.front() };
        std::ifstream               ifs{ "document.json.w" };
        cereal::JSONInputArchive  iarchive{ ifs };

        iarchive( d ); // Read the data

        std::wstring expected = L"Das Verstehen, die Meinung, fällt aus unsrer Betrachtung heraus.";

        /// Make sure to properly imbue!
        std::wcout << L"[" << d.contents() << L"]" << std::endl;
        std::wcout << L"[" << expected     << L"]" << std::endl;

        EXPECT( d.filename() == f.wfilenames.front() );
        EXPECT( d.contents() == expected );
    },

    "can serialize (automaton with added document) [as std::wstring]", []
    {
        sis::Document<std::wstring> d{ f.wfilenames.front() };
        std::ofstream               ofs{ "document.bin" };
        cereal::BinaryOutputArchive oarchive{ ofs };

        oarchive( d ); // Write the data to the archive
    },

    "can deserialize (automaton with added document) [as std::wstring]", []
    {
        sis::Document<std::wstring> d{ f.wfilenames.front() };
        std::ifstream               ifs{ "document.bin" };
        cereal::BinaryInputArchive  iarchive{ ifs };

        iarchive( d ); // Read the data

        std::wstring expected = {L"Das Verstehen, die Meinung, fällt aus unsrer Betrachtung heraus."};

        /// Make sure to properly imbue!
        // std::wcout << L"[" << d.contents() << L"]" << std::endl;
        // std::wcout << L"[" << expected     << L"]" << std::endl;

        EXPECT( d.filename() == f.wfilenames.front() );
        EXPECT( d.contents() == expected );
    },

};

int main() {
    std::setlocale(LC_ALL, "");
    return lest::run( specification );
}

template <class F = std::ofstream, class A = cereal::BinaryOutputArchive>
void save(std::string & fn /* fs::path & p */);

template <class F = std::ifstream, class A = cereal::BinaryInputArchive>
static
DocumentIndexingAutomaton<AutomatonType>
load(std::string & fn);

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

namespace sis {

/*****************************************************************************/
template <class AutomatonType>
template <class FileType, class ArchiveType>
inline
void
DocumentIndexingAutomaton<AutomatonType>::
save(std::string & fn /* fs::path & p */)
{
    FileType        ofs{ fn };
    ArchiveType     oarchive{ ofs };

    index();
    automaton_->save(fn + ".c");     // Write automaton information to archive
    oarchive( *this );               // Write the document data to the archive
}


/*****************************************************************************/
template<class AutomatonType>
template<class FileType, class ArchiveType>
inline
DocumentIndexingAutomaton<AutomatonType>
DocumentIndexingAutomaton<AutomatonType>::
load(std::string & fn)
{
    FileType            ifs{ fn };
    ArchiveType         iarchive{ ifs };

    auto aut = Factory::create<AutomatonType>();
    aut->load(fn + ".c");

    DocumentIndexingAutomaton<AutomatonType> a;
    iarchive( a );
    // a.automaton_ = std::shared_ptr<AutomatonAbstract<typename AutomatonType::string_type>>(aut);
    a.automaton_ = aut;
    for (auto& index: *(a.index_)) {
        index.second.set_voidsequence(a.automaton()->data());
    }
    return std::move(a);
}

/* .... */

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

"Can write to disk in JSON format to std::ofstream", []
{
    std::cout << "saving to json" << std::endl;
    refresh_indexer();
    idx->save<std::ofstream,cereal::JSONOutputArchive>(f.filename.json);
},

"Can read back from disk in JSON format from std::ofstream", []
{
    std::cout << "loading from json" << std::endl;
    auto index_load = sis::DocumentIndexingAutomaton<sis::SCDAWGAdapter<std::string>>::load<std::ifstream,cereal::JSONInputArchive>(f.filename.json);
},

"Can write to disk in binary format to std::ofstream", []
{
    std::cout << "saving to binary" << std::endl;
    refresh_indexer();
    idx->save<std::ofstream,cereal::BinaryOutputArchive>(f.filename.binary);
},

"Can read back from disk in binary format from std::ofstream", []
{
    std::cout << "loading from binary" << std::endl;
    auto index_load = sis::DocumentIndexingAutomaton<sis::SCDAWGAdapter<std::string>>::load<std::ifstream,cereal::BinaryInputArchive>(f.filename.binary);
    // assert(aut.automaton_ != nullptr);
},

"Can write to disk in binary format to std::ofstream (with default template arguments)", []
{
    refresh_indexer();
    idx->save<>(f.filename.binary);
},

"Can read back from disk in binary format from std::ofstream (with default template arguments)", []
{
    auto aut = sis::DocumentIndexingAutomaton<sis::SCDAWGAdapter<std::string>>::load<>(f.filename.binary);
    // assert(aut.automaton_ != nullptr);
},

"Can write to disk in binary format to std::ofstream (with default template arguments -- and without using `<>')", []
{
    refresh_indexer();
    idx->save(f.filename.binary);
},

/* ... */

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

template<class AutomatonType>
struct DocumentIndexingAutomaton
{

    /* .... typedefs für die member-Typen AutomatonAbstract_t, etc ...*/

    AutomatonAbstract_t                     automaton_ = nullptr;
    std::unique_ptr<DocumentCollection_t>   documents_ = nullptr;
    std::unique_ptr<DocumentIndex_t>        index_     = nullptr;
    std::map<UINT, std::vector<Document_t>> states_    = nullptr;
    FindResult_t                            results_   = nullptr;
    size_t                                  total_length_ = { 0 };
    bool                                    indexed_   = false;
    string_type                             query_;

    template <class Archive> void serialize( Archive & archive ) {
        archive(
                CEREAL_NVP(documents_),
                CEREAL_NVP(index_),
                CEREAL_NVP(states_),
                CEREAL_NVP(indexed_)
        );
    }

3.6.2 Web-Ebene

In diesem Kapitel werden die wesentlichen Bausteine der Web-Ebene von SIS besprochen. Der Web-Teil ist im wesentlichen eine SPA (Singe Page App) und baut hierbei weitestgehend auf den sogenannten MEAN-Stack¹⁵ auf, lässt aber u.a. die Datenbank weg, da die Daten von den Automaten in SIS geliefert werden.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

## --[ app.coffee ]-- ##

# import
indexer = require './build/Release/sis'
log.info('index imported', {})

# deserialize
indexerObj = indexer.load '../etc/allfiles'
log.info('index loaded', {})

# size
size = indexerObj.size()
log.info('index size', {size: size})


## --[ query.coffee ]-- ##
# query
found = app.get('indexer').find(query, width)

17
18
19
20
21

if($ENV{sis_BUILD_WEB_COMPONENT})
    set(sis_BUILD_WEB_COMPONENT      ON)
else()
    set(sis_BUILD_WEB_COMPONENT     OFF)
endif()

[...]
-- [WEB COMPONENT]
--   Will NOT build web component
[...]

[...]
-- [WEB COMPONENT]
--   Will build web component
[...]

sis/web> npm install

sis/web> bower install

web/sis> node-gyp rebuild

{
  "targets": [
    {
      "target_name": "hello",
      "sources": [ "hello.cc" ]
    }
  ]
}

{
    'targets': [
    {
        'target_name': 'sis',
        'sources': [
            'cpp/sis.cpp',
            'cpp/DocumentIndexingAutomaton.cpp'
        ],
        'include_dirs': [
            '/usr/include',
            '<!@(echo `pwd`/../src/)',
            '<!@(echo `pwd`/../vendor)',
            '<!@(echo `pwd`/../vendor/serialization/include)',
            '<!@(echo `pwd`/../vendor/make_unique)',
            '<!@(echo `pwd`/../vendor/pugixml/src)'

        ],
        # global flags go here
        # `cflags_cc!` is supposed to mean: remove these flags
        'cflags_cc!': [ '-fno-rtti' ], # '-fno-exceptions'
        # `cflags_cc` is supposed to mean: add these flags
        'cflags_cc':  [ '-fPIC', '-std=c++11', '-fexceptions', '-Wall', '-Wextra', '-Wno-attributes', '-Wno-pointer-arith' ],
        'conditions': [
    /* [......] */ 

3.6.3 Node Addon

3.6.4 Nächste Schritte

• Alternativ zu einem Node-Addon gibt es die Möglichkeit, durch die Verwendung von emscripten (Emscripten Authors 2013), C++ nach Javascript zu kompilieren. Allerdings müssten dabei die kompletten Index-Dateien in den Browser des Client übertragen werden, weshalb hier nur der Hinweis auf emscripten gegeben werden soll.

4.1.1 Dateistruktur

.
|-- Makefile                      # targets zum starten, stoppen, restart
|-- README.md
|-- bower.json                    # ini-file für client-seitige dependencies
|-- bower_components              # Ordner für client-seitige dependencies
|-- database                      # MongoDB-Anbindung
|-- node_modules                  # Ordner für Server-seitige dependencies
|-- npm-shrinkwrap.json           # ini-file für server-seitige dependencies (unwichtig)
|-- package.json                  # ini-file für server-seitige dependencies (wichtig)
|-- public                        # client-seitiger Applikationsteil: Application-Logic (AngularJS)
|-- routes.js                     # Server-seitige Routes
|-- views                         # client-seitiger Teil: Views (Jade)
`-- wast-feedback-server.js       # Der Web-Server selbst (node)

10.2.1 Motivation

cmake zu verwenden bringt mehrere Vorteile:

1. die Syntax von cmake ist einfacher als die von “klassischen” Makefiles
2. dadurch, dass die CMakeLists.txt-Dateien rekursiv aufeinander aufbauen und jeweils an geeigneten Stellen im Projekt untergebracht werden, bleiben diese stets übersichtlich

10.2.2 vs. autotools (autotool-hell)

cmake wird vor allem deshalb gerne verwendet, weil der “klassische” autotools-Weg sehr umständlich, fehleranfällig und schlicht “altbacken” sein kann – siehe hierzu den Ablaufgraph “Autotools”, welcher den autotools-Weg nachzeichnet.

Autotools

autotools fasst wiederum den bis dahin verwendeten Weg und die dazu benötigten tools zusammen, die benutzt wurden um Makefiles generieren zu lassen. Den autotools-Weg erkennt man immer daran, dass eine Software den “klasssischen Dreischritt” ./configure && make && make install verwendet.

10.2.3 out-of-source-builds

Klassischerweise verwendet man bei cmake in sog. “out-of-source builds”. Einfach gesagt, heisst dies nur, dass man seinen Build nicht direkt in der source macht, sondern dafür ein eigenes Verzeichnis (meist build/) anlegt, und fortan von dort arbeitet.

Damit vermeidet man das “Vermüllen” des source-directories mit Object-Files und dergleichen und hält so die Quellen stets sauber.

Der Workflow ist also:

~/project/foo > mkdir -p build && cd $_   # Hier liegt u.a. src/ mit den Quellen und (mindestens) einer CMakeLists.txt
~/project/foo/build > cmake ..            # `..` ist übergeordnetes Verzeichnis!
~/project/foo/build > make                # build-files werden out-of-source angelegt!

10.2.4 Wichtige optionen für cmake

Die beiden witchtigsten Optionen für cmake sind:

• -DCMAKE_CXX_COMPILER=/opt/local/bin/g++ # Verwende speziellen Compiler
• -DCMAKE_BUILD_TYPE=Debug # Build type festlegen (andere Switches für den Compiler)

Diese gibt man cmake mit, das dann entsprechend die Makefiles generiert:

cmake -DCMAKE_CXX_COMPILER=/Users/dbruder/bin/g++ -DCMAKE_BUILD_TYPE=Debug ..

10.2.5 Beispiel

Hier ein Auszug aus den CMakeLists.txt-Dateien des Projekts sis:

project(sis)
cmake_minimum_required(VERSION 2.8.3)
set(sis_AUTHORS "\"Daniel Bruder <bruder@cip.ifi.lmu.de>\"")

set(sis_NAME     "\"SIS\"")
set(sis_SUBNAME  "\"Symmetric Index Structures\"")
set(sis_FULLNAME "\"SIS -- Symmetric Index Structures\"")

# version numbers: lassen sich gut in doxygen etc verwenden
set(sis_VERSION_MAJOR 3)
set(sis_VERSION_MINOR 8)
set(sis_VERSION_PATCH 0)

set(CMAKE_INSTALL_PREFIX            /usr/local)
set(FIND_LIBRARY_USE_LIB64_PATHS)

# diese Variablen können auf der Commandline mit -D gesetzt werden
set(sis_BUILD_TESTS                  ON)
set(sis_TEST_STATIC_ASSERTIONS        0) # 0 == 'OFF' / 0 == 'ON'
set(GIT_SUBMODULES_CHECKOUT_QUIET    ON)

set(CONF_DIR ${PROJECT_SOURCE_DIR}/cmake)

include(${CONF_DIR}/config.cmake)
include(${CONF_DIR}/compilerdetect.cmake)
include(${CONF_DIR}/processorcount.cmake)
include(${CONF_DIR}/buildflags.cmake)
include(${CONF_DIR}/doxygen.cmake)
include(${CONF_DIR}/boost.cmake)
include(${CONF_DIR}/ctest.cmake)
include(${CONF_DIR}/cpack.cmake)
include(${CONF_DIR}/host.cmake)
include(${CONF_DIR}/checklocale.cmake)
include(${CONF_DIR}/submodules.cmake)
include(${CONF_DIR}/githooks.cmake)
include(${CONF_DIR}/nodewrap.cmake)

add_subdirectory(src/)
add_subdirectory(src/features)
add_subdirectory(vendor/)
add_subdirectory(web/)

Im Verzeichnis src/ findet sich folgendes cmake-file:

# Diese Verzeichnisse mit -I an den Compiler als Suchpfade mitgeben
include_directories(${CMAKE_SOURCE_DIR}/src/)

# Diese Verzeichnisse mit -isystem (oder nach OS abhängig) an den Compiler als Suchpfade mitgeben. -isystem bindet "externe" Header so ein, dass diese keine Warnings werfen.
include_directories(SYSTEM
    ${CMAKE_SOURCE_DIR}/vendor/
    ${CMAKE_SOURCE_DIR}/vendor/cautomata
    ${CMAKE_SOURCE_DIR}/vendor/make_unique
    ${CMAKE_SOURCE_DIR}/vendor/serialization/include
)

# Shared lib bauen
add_library(cppautomataadapter SHARED
    adapter/CompressedAutomatonAdapter.cpp
    adapter/SCDAWGAdapter.cpp
    adapter/VoidSequenceAdapter.cpp
)
target_link_libraries(cppautomataadapter cautomata)


add_library(cppindexer SHARED
    indexer/DocumentIndexingAutomaton.cpp
)
target_link_libraries(cppindexer cppautomataadapter cautomata)

add_executable       (sis main.cpp)
target_link_libraries(sis cppautomataadapter cppindexer cautomata ${Boost_LIBRARIES})

11.1.1 Schematische Darstellung

Web Apps Basics

11.1.2 Unterschiedliche Kombinationsmöglichkeiten

Hier eine kleine Liste, was an welchen Stellen möglicherweise vorkommen könnte – diese Liste ist bei weitem nicht exhaustiv!

11.2.1 Vorteile

Entscheidender Vorteil des MEAN-Stacks gegenüber Ruby on Rails sind u.a. folgende

• man kann durch alle Teile hinweg stets nur in Javascript bleiben
• die Webseiten sind wesentlich performanter und die Load auf dem Server bleibt wesentlich geringer
• die Sub-Komponenten (die gleich besprochen werden) sind jeweils gegen andere austauschbar: in diesem Sinne handelt es sich um einen Stack: man kann für die jeweiligen Aufgaben durchaus auch andere Module zum Einsatz bringen
• es gibt weniger Konventionen und einen freieren Aufbau (dies kann sowohl ein Vor- als auch ein Nachteil sein)

Hierbei stehen die Initialen für folgende Teile, welche – ganz grob umrissen –, folgende Aufgaben übernehmen:

• MongoDB²⁷ – Datenbank
• Express²⁸ – Framework aufbauend auf Node, für die Server-Seite
• Angular²⁹ – für die Client-Seite
• Node³⁰ – der Webserver (ähnlich wie, aber statt Apache, nginx, unicorn, …)

Wie oben angedeutet ließen sich die einzelnen Komponenten des Stack auch durch andere Komponenten austauschen – dazu im folgenden eine kleine Aufzählung, welche Alternativen (unter vielen vielen mehr) potentiell bestehen könnten – hierbei werden vielleicht auch die Aufgaben der einzelnen Komponenten deutlicher:

• MongoDB: mysql, postgresql, redis, couchDB, …
• Express: koa, pures Node, …
• Angular: jQuery + php, ember.js, sails.js, …
• Node: Apache, nginx, unicorn, foreman, …

11.2.2 Node

Node³¹ wird verwendet um einen eigenen Webserver selbst zu schreiben. Node ist ein unglaublich schneller Webserver und ein funktionsfähiger eigener Webserver lässt sich in Node bereits mit wenigen Zeilen schreiben.

Da am Ende Webseiten über eine normale WWW-Adresse ausgeliefert werden sollen und meistens ein Apache auf dem Server bereits läuft und auf dem HTTP-Port 80 “listened”, kann dieser Port nicht erneut belegt werden. Node macht daher seinen eigenen Port auf, z.B. 6688, was aber wiederum bedeuten würde, dass der user die seite auf diesem Port ansteuern müsste, z.B. webseite.com:6688.

Für diesen Fall wird zumeist ein sog. “Reverse Proxy” im Apache eingerichtet, der alle unterschiedlichen Services und Webserver die möglicherweise sonst noch auf dem Server laufen zusammenfasst und eine einheitliche Oberfläche nach außen bietet (d.h. keine expliziten Port-Angaben vom User verlangt, die ihn ohnehin nur verwirren würden). Bei einem Reverse-Proxy wird Apache mitgeteilt bestimmte Pfade auf bestimmte Ports einfach weiterzuleiten und nicht selbst zu bearbeiten.

11.2.3 Angular

Angular erlaubt es, vieles an Logik bereits in der View zu lösen, d.h. an der Stelle, an der es in einem MVC-Pattern³² häufig gebraucht wird. (Der andere, gegenteilige Ansatz nennt sich “Fat Controller”).

Angular erlaubt das ganz hervorragende “double data-binding”, welches man sich lieber live in einem Beispiel anschaut um es zu verstehen. Damit erlaubt es Angular unglaublich schnell schöne Web-Applikationen zu entwickeln.

11.2.4 Express

Express ist ein Mini-Framework, das auf Node aufbaut, und es deutlich einfacher macht, einen Server mit Routes, Middleware, etc auf die Beine zu stellen. Dabei arbeitet es sehr eng mit dem Node-Webserver zusammen.

11.2.5 MongoDB

MongoDB (von “humounguous”, riesig) ist eine sogenannte “NoSQL”-, bzw. Dokumentenorientierte Datenbank.

Dabei speichert es im Gegensatz zu SQL-artigen Datenbanken JSON-Objekte (“Dokumente”) anstatt von Tabellen und ist wesentlich einfacher zu bedienen.

Auch passt es sich besser an sich schnell und häufig wechselnde Datenbank-Schemata an und ist dadurch wesentlich flexibler als eine SQL-Basierte Datenbank.

11.3.1 Bootstrap

11.3.2 Bootstrap UI

11.3.3 jQuery

13.1.1 Hinweise zur Installation der dependencies

Für Hinweise zur Installation der dependencies bezüglich der Rechner am CIP-Pool, vergleiche die README.md in BA Thesis Template⁵⁰.

13.1.2 Hinweise zur Benutzung des Makefiles

Für Hinweise zur Benutzung des Makefiles, vergleiche ebenfalls die README.md in BA Thesis Template⁵¹.

14.1.1 Bündelung der Komponenten unter einem Dach

Die zentrale Stelle an der alle Komponenten der WAST-Landschaft zusammenlaufen ist das repository wittfind-web.

wittfind-web stellt nach außen hin die zentrale Webseite als Einstieg für die Nutzer dar; nach innen hin ist dies der Schnittpunkt an dem alle Module zusammenlaufen⁵².

Das zentrale Makefile, welches alle diese administrativen Aufgaben übernimmt (und dabei u.a. an komponentenspezifische Makefiles weiterleitet ist hier).

+++ BEGIN DEPRECATED +++

Ab hier ist dieses Kapitel ein bisschen out-dated, vermittelt aber trotz allem den richtigen Eindruck von unserem neueren Makefile-Ansatz, bitte die folgenden Teile Updaten!

Um die Komponenten und Tools unter einem Dach zu bündeln bietet wast-infrastructure den Befehl wast.

Unter wast werden die verschiedenen Komponenten registriert und sind damit für den “unified deployment”-Prozess verfügbar:

> ./wast
WAST, version 0.9.5

This is the help section

Usage:

  ./wast <command>

Available COMMANDS:
init                    : initialize (all) components
build <component>       : build component <component>
start <component>       : start component <component>
stop  <component>       : stop component <component>
restart <component>     : restart component <component>

Available COMPONENTS:
wast-feedback-app
wast-website
wf
sis

Die Komponenten werden unter Available COMPONENTS gelistet und sind somit für die Befehle, die unter COMMANDS gelistet sind, verfügbar.

Beispielsweise lässt sich somit also die Komponente wf mit folgendem Befehl bilden und starten:

> ./wast init wf  && \
  ./wast build wf && \
  ./wast start wf

Selbstverständlich lässt es sich darüber nachdenken, welche weiteren Sub-Befehle für wast noch denkbar sind (z.B. ist wast release <version> noch nicht implementiert, und denkbar sind auch Befehle wie wast test <component>, wast deploym <component> um die vorhergehenden drei Befehle zusammenzufassen). Dieser Teil befindet sich bisher noch in der beta- und Evaluations-Phase, daher sollen zunächst die wichtigsten Befehle robust implementiert sein, bevor neue Funktionalität hinzugefügt wird.

Um wast neue Funktionalität hinzuzufügen oder eine Komponente Teil des unified deployment-Prozesses zu machen ist gar nicht viel nötig, wie im nächsten Teil gezeigt wird.

14.1.2 Abstraktion des deployment-Prozesses

wast erwartet von einer Komponente im wesentlichen 2 Dinge:

• die Komponente woll in wast registriert sein
• die Komponente soll für alle wast-Sub-Befehle, die unterstützt werden sollen, ein entsprechendes Skript liefern. Konkret muss wf, wenn es die Befehle wast build und wast start anbieten will, in seinem Verzeichnis die Dateien wast.build und wast.start anbieten.

14

COMPONENTS=(wast-feedback-app wast-website wf sis)

#!/bin/sh

set -x  # Show the commands that are executed

# try "Creating build directory"      by 
mkdir -p build

# try "Switching to build directory"  by 
cd build

# try "Running cmake"                 by 
cmake ..

# try "Running make"                  by 
make VERBOSE=1

# try "Switching to web component"    by 
cd ../web

# try "Installing web dependendices"  by 
CXX=/usr/bin/g++-4.8 npm install

# try "Installing web dependendices"  by 
bower install

#!/bin/sh
set -x
# try "Starting feedback app" by 
cd web
LD_LIBRARY_PATH=/srv/www/vhosts/wast/components/sis/build/lib \
sudo -u wastd forever start -a app.coffee

15.1.1 Vorteile

Die entscheidenden Vorteile des Wrappens von C nach C++ sind:

• automatische und saubere Allokation / Deallokation von Klassen bzw. structs (dieses ist auch bekannt unter dem Namen “RAII” – zu diesem siehe auch das nächste Kapitel sowie Definitionen)
• einfacheres “Handling” der Instantiierung von Klassen, also Konstruktion und, v.a. sauberere, sicherere Destruktion

Selbstverständlich lassen sich nach dem Wrapping auch alle Vorteile von C++ bzw. C++11 (und, später C++14) nutzen. Darunter fallen u.a.:

• Die Smart Pointers shared_ptr und unique_ptr: Diese legen u.a. feste Ownership zu Grunde. Damit wird verhindert, dass man vergisst ein struct wieder zu deallokieren und die Zuständigkeit wird festgelegt: wer soll den pointer am Ende wieder deallokieren?

• Templatisierung

• Vererbung und Interfaces und damit klare Strukturen: Mit C++ lässt sich viel stärker auf der Problem-Ebene arbeiten denn auf der reinen Sprach-Ebene

15.1.2 Patterns

Grundlegend funktioniert das Adapter/Wrapper-Pattern immer auf die gleiche Art und Weise: Es gibt eine Klasse die gewrapped werden soll, den Adaptee und einen Wrapper, den Adaptor bzw. Wrapper.

Adapter Pattern

Der Wrapper hält eine Verbindung zum Adaptee und bildet die API nach. Der Client wird nur noch einen Adaptor bzw. Wrapper instantiieren und der Adaptor kümmert sich darum, den entsprechenden, eigentlichen Adapee zu konstruieren.

Wenn der Client Methoden des Adaptors verwendet, so wird dieser lediglich die Parameter an den Adaptee weiterleiten und die Ergebnisse, die vom Adaptee kommen wieder an den Client zurückleiten.

Im Fall von C++ bietet es sich natürlich an, den Adaptee als Pointer im Adaptor zu halten.

Ein mögliches, elegantes, sicheres und eigentlich simples Pattern um C structs nach C++ zu wrappen beschreibt (R. Martinho Fernandes 2013) mit seiner “Rule of Zero”. Eine konkrete Anwendung wird im Kapitel ab Seite sowie hier im Detail beschrieben.

Um dieses Pattern besser verstehen zu können soll an dieser Stelle noch einmal ein genuerer Blick darauf geworfen werden. Als Beispiel dient in diesesm Fall die Klasse VoidSequenceAdapter, welche das C-Struct VoidSequence wrapped.

VoidSequence stellt in der C-Schicht eine Art std::vector dar, indem es eine Liste von Elementen führt, und, falls der allokierte Speicher nicht ausreicht, neuen besorgt, um weitere Elemente in seinen Members einfügen zu können. Dabei speichert es beliebige Objekte, indem es sich als void* verwaltet – letzteres setzt natürlich voraus, dass diese pointer später wieder entsprechend “zurückgecastet” werden.

Wie auch immer das struct im Detail aussieht, das Wrapping-Pattern bleibt stets identisch. Zunächst soll ein naiver Ansatz vorgestellt werden, welcher dann kontrastiert wird von dem “Rule of Zero”-Ansatz.

1
2
3
4
5
6
7

#include "Adaptee.h"

class Adaptor {
  Adaptee*   adaptee_;
  Adaptor(): adaptee_{ new Adaptee() } {};
  ~Adaptee() { delete adaptee_; }
};

Dieser naive Ansatz zeigt das Wrapping-Pattern in klassischer Weise in C++: es wird ein Pointer zum Adaptee gehalten. Der Pointer wird im Konstruktor hergestellt und im Destruktor deallokiert. Problematisch wird es, wenn der “Konstruktor” des Adaptee eine exception wirft: Konstruktoren sollen keine Exceptions werfen, da es sonst schwierig wird, bereits gelaufene Allokationen rückgängig zu machen.

Ein fortgeschrittenerer Ansatz ist der “Rule of Zero”-Ansatz von (R. Martinho Fernandes 2013): In diesem Fall ist auch die Ownership zur Adaptee-Ressource geregelt, da es sich hier um einen unique_ptr handelt.

1
2
3
4
5
6
7
8
9
10
11
12
13

#include "voidSequence.h"

class VoidSequenceAdapter {
  using adaptee_handle = std::unique_ptr<VoidSequence, decltype(&::VoidSequenceFree)>;
  
private:
  adaptee_handle adaptee_;
  
public:
   VoidSequenceAdapter()
   : adaptee_{ ::VoidSequenceInit(1), &::VoidSequenceFree } {}
  ~VoidSequenceAdapter() {}
};

15.2.1 Beispiel

Um das Beispiel aus dem vorherigen Kapitel erneut aufzugreifen, soll an dieser Stelle auf die tatsächliche Implementation des adaptee_handle eingegangen werden:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

class Managed : std::true_type {};
class Unmanaged : std::false_type {};

template<class StringType = std::string, class ManagedAdaptee = Managed> class VoidSequenceAdapter;

template<class StringType, class ManagedAdaptee>
class VoidSequenceAdapter {

using adaptee_handle = typename
    std::conditional<
      ManagedAdaptee::value == true                                 // if adaptee is to be managed by this adapter
    , std::unique_ptr<VoidSequence, decltype(&::VoidSequenceFree)>  // then handle is a unique_ptr w/ custom DTOR
    , VoidSequence*                                                 // otherwise: bare ptr, no automatic deletion.
>::type;                                                            // finally: don't forget to choose this type
adaptee_handle adaptee_;

/* ... */
};

Abegesehen von den Template Meta Programmierungs-Anteilen (std::conditional) und den Type Traits-Anteilen (ManagedAdaptee::value) in diesem Beispiel soll an dieser Stelle der Policy-Based Ansatz (ManagedAdaptee) genau betrachtet werden.

Kurz zur Wiederholung: Nach der Defintion in Zeile 4, kann ein VoidSequenceAdapter auf folgende Arten und Weisen instantiiert werden und dabei ein sehr unterschiedliches Verhalten aufweisen:

1
2
3
4

VoidSequenceAdapter<>                        v1; /* std::string, managed     */
VoidSequenceAdapter<std::wstring>            v2; /* std::wstring, managed    */
VoidSequenceAdapter<std::wstring, Managed>   v3; /* same                     */
VoidSequenceAdapter<std::wstring, Unmanaged> v4; /* std::wstring, unmanaged! */

Wie im vorherigen Kapitel besprochen, wird die Klasse VoidSequenceAdapter – im Standardfall – den Speicher des Structs adaptee_ automatisch am Ende seiner eigenen Lifetime deleten, indem es die Funktion VoidSequenceFree aus der C-Schicht aufrufen wird.

Nun kann es aber ganz bestimmte Situationen geben, in welchen dieses Verhalten nicht nur unerwünscht sondern fehlerhaft wäre. In diesen Fällen lässt sich ein weiteres Template-Argument, Unmanaged mitgeben, um die Klasse derart anders zu gestalten, dass sie kein (!) automatisches delete von VoidSequence ausführen wird.

Dabei spielen folgende Teile zusammen:

• class Managed : std::true_type {}; ist eine bereits fertige Klasse ohne weitere Implementation. Sie stellt einen “sprechenden” strong type dar, der sich als Template-Argument nutzen lässt. std::true_type stellt einen Typ aus C++11 dar, der folgendem Typ entspricht: std::integral_constant<bool, true>. Das heisst, es handelt sich hier um einen “strong type” vom Typ bool mit der Belegung true, dessen einfachste Implementierung folgende sein könnte (der Standard übernimmt dieses aber):

  template<class T, T v>
  struct integral_constant {
      static constexpr T value = v;
  };

• Hiermit wiederum lassen sich entscheidende Weichen (zur Compile-Zeit!) stellen, um VoidSequenceAdapter entweder Managed oder Unmanaged zu machen. Um zur Compile-Zeit die Klasse VoidSequenceAdapter anders zu konfigurieren lässt sich Managed::value abfragen, um zu sehen, wie die Klasse konfiguriert werden soll (vergleiche hierzu das “Type Traits”- sowie das “Template Meta Programmierungs”-Kapitel).

• In diesem elaborierteren Fall wird also der Typ des adaptee_handle zur Compile-Zeit entschieden und kann – je nach Typ von ManagedAdaptee – eines der beiden folgenden werden:

  std::unique_ptr<VoidSequence, decltype(&::VoidSequenceFree)>

  oder schlicht:

  VoidSequence*

Mit diesem Policy-Based Ansatz ist es also möglich, eine Klasse zu schreiben, die mehreren Ansprüchen – je nach Bedarf – gerecht werden kann: Im Standardfall ist VoidSequenceAdapter eine Adapter-Klasse, welche den RAII-Prinzipen folgt; sollte jedoch ein anderes, möglicherweise orthogonales, Verhalten benötigt werden, so lässt sich dieses ebenso umsetzen und der restliche Code, der VoidSequenceAdapter nutzt, kann diesen auch weiterhin wie gewohnt nutzen.

Zu diesem Kapitel wird empfohlen auch die anderen Kapitel Template Meta Programmierung, SFINAE, Type traits, und Silent Degredation genau zu beachten; vor allem aber (Alexandrescu 2000), (Di Gennaro 2012) und (Abrahams and Gurtovoy 2005) genau zu studieren.

15.3.1 Beispiel

/// @brief meta-class to handle StringType type traits (using static assertions)
template<class T>
struct ElementSize {
    static constexpr bool T_is_string_type =
                std::is_base_of< std::string, T>::value
             || std::is_base_of<std::wstring, T>::value ;

     static_assert(T_is_string_type,
         "\n\n\tYou must instantiate this class in question with std::string,"
         "\n\tstd::wstring, or any class derived thereof.\n"
     );


    typedef T                               type;
    typedef T                        string_type;
    typedef typename T::value_type    value_type;

    typedef typename std::conditional<
        std::is_same<std::string,T>::value, // if StringType is std::string
        std::fstream,                       // then stream_type is std::fstream
        std::wfstream                       // else it is std::wfstream
        >::type                             stream_type;

    static constexpr unsigned int value =
                            sizeof(typename string_type::value_type);
};

15.6.1 Beispiel

Angenommen man hat eine Klasse DoItAll mit folgender implementation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

template <class Type>
class DoItAll {
  Type member_;

public:

  /* CTOR, DTOR */
  DoItAll(Type t): member_{t} {}
  ~DoItAll() {}

  /* Methods */
  inline Type plus(Type other) const { return member_ + other; }
  inline Type div(Type other) const { return member_ / other; }
};

Angenommen, DoItAll würde folgendermaßen genutzt werden:

1
2
3
4

DoItAll<int>  doitall_int{8};
  
std::cout << doitall_int.plus(34) << std::endl; // 42
std::cout << doitall_int.div(4) << std::endl;   // 2

Oder auch so:

1
2
3

DoItAll<std::string>  doitall_str{"this is"};
  
std::cout << doitall_str.plus(" a string") << std::endl; // "this is a string"

…so wäre bis hierhin alles in Ordnung und würde sauber kompilieren – trotz dessen, dass std::string / std::string ill-formed und kein gütliger Aufruf wäre!

Das Geheimnis dahinter ist, dass Methoden von Template Klassen erst in dem Moment konkret instantiiert werden, in dem sie auch tatsächlich genutzt werden, d.h. wenn an keiner Stelle doitall_str.div("some string") aufgerufen wird, wird diese Methode auch nicht instantiiert und die Klasse bleibt kompilierfähig.

Dies ist das Geheimnis hinter silent degredation bzw. “incomplete instantiation” und erlaubt wiederum sehr flexible Klassen zu gestalten.

15.6.2 Bemerkung

Mit C++11 gibt es eine Möglichkeit, Kompilierzeit zu sparen, indem man den Compiler mit extern template anweisen kann, sich darauf zu verlassen, dass eine volle Instantiierung/Spezialisierung einer Klasse an anderer Stelle vorgenommen wird und an dieser Stelle nicht zu instantiieren.

Im Normalfall wird nämlich die template class X überall dort, wo sie verwendet wird, vom Compiler (u.U. auch viele Male) instantiiert nur um am Schluss alle diese – bis auf eine – wieder zu verwerfen.

C++11 erlaubt eine einmalige Spezialisierung auf die sich andere Code-Teile stützen können mit:

extern template class std::vector<MyClass>;

Allerdings beisst sich nun dieses Konzept mit der “silent degredation” da durch dieses Vorgehen eine volle Spezialisierung und damit eine volle Instantiierung vorgenommen wird.

16.1.1 Hinweise

16.3.1 Beobachtungen

• Der erste Character belegt 1 Byte
• unsigned int beträgt 4 Byte (auf einem 64-Bit-System)
• bool belegt 1 Byte: Obwohl ein bool theoretisch nur 1 Bit statt einem ganzen Byte (8 bit) bräuchte, tritt hier das Phänomen des Padding auf und das bit wird aligned (siehe dazu http://en.wikipedia.org/wiki/Data_padding)
• Der std::vector<std::string> ist folgendermaßen aufgebaut:
  • das erste byte wird belegt mit der Angabe, wie viele Elemente der vector enthält, also mit vector.size()
  • das nächste byte kündigt die Größe des ersten std::strings an: 4
  • nun folgen die 4 characters des ersten strings
  • da der vector 2 strings enthält muss nun die Längeninformation über den zweiten String folgen: string2.size() == 6
  • nun folgen noch 6 chars des zweiten strings des vector und die Informationen sind vollständig.

16.4.1 Boost.Serialization

Vergleiche http://www.boost.org/doc/libs/1_55_0/libs/serialization/doc/index.html

16.4.2 cereal

Die API von cereal ist relativ einfach – für einen tiefergehenden Einblick vergleiche das Kapitel Zweistufige Serialisierung in SIS: Symmetric Index Structures.

Im Grunde ist es lediglich notwendig eine Methode der folgenden Art in seiner Klasse zu schreiben:

Angenommen, die Klasse hat folgendes Member list_:

#include <vector>
#include <string>

class WordList {
public:
    typedef std::vector<std::string> list_t;

    /* Constructors, Methods, etc... */

private:            // Members
    list_t list_;
};

So reicht es bereits aus, die notwendigen header von cereal und eine Methode serialize() folgendermaßen zu schreiben:

// This method lets cereal know which data members to serialize
template<class Archive>
void serialize(Archive & archive) {
    archive( list_ ); // serialize things by passing them to the archive
}

Die Klasse sieht nun also so aus:

#include <vector>
#include <string>

#include <cereal/types/string.hpp>
#include <cereal/types/vector.hpp>

class WordList {
public:
    typedef std::vector<std::string> list_t;

    /* Constructors, Methods, etc... */

private:    
    list_t list_;

public:
    template<class Archive>
    void serialize(Archive & archive) {
        archive( list_ );
    }
};

Nun kann die Klasse von aussen serialisiert und deserialisiert werden:

#include <iostream>
#include <fstream>
#include <string>

#include <cereal/archives/binary.hpp>

#include "WordList.hpp"

int main() {
    WordList wl;

    // Elemente zum Container hinzufügen
    std::string baz{"baz"};
    wl.add("foo");
    wl.add(std::string{"bar"});
    wl.add(baz);

    std::cout << wl.size() << std::endl;

    // Binär-Datei anlegen, in welche serialisert werden soll
    std::ofstream os{"out.bin", std::ios::binary};
    
    // cereal-Archiv auf die Binär-Datei anlegen
    cereal::BinaryOutputArchive binary_archive_file{ os };
    
    // Serialisierbaren Container in das cereal-Archiv geben
    binary_archive_file( wl );

    return 0;
}

Nach XML oder JSON zu serialiseren funktioniert genau nach dem selben Schema:

int main() {
    /* Container füllen ... */

    // JSON nach std::cout serialisieren:
    cereal::JSONOutputArchive     json_archive_cout{std::cout};
    json_archive_cout( wl );
    
    /* ... */
}

Das Einlesen ist wiederum genauso einfach:

#include <iostream>
#include <fstream>
#include <string>

#include <cereal/archives/binary.hpp>

#include "WordList.hpp"

int main() {
    
    /* Es gibt eine Binär-Serialisierte WordList ... */
    std::ifstream is{"out.bin", std::ios::binary};

    // cereal Input-Archiv auf diese Datei festlegen
    cereal::BinaryInputArchive binary_archive{ is };

    // leeren Container anlegen
    WordList wl;
    
    // Container aus Binär-Datei laden
    binary_archive(wl);

    return 0;
}

16.4.3 generic serialize

generic serialize erlaubt es, beliebige Standard-Container (mit beliebiger Schachtelung) individuell mit save zu speichern und mit load zu laden. Es kümmert sich automatisch darum, die richtige Speichermethode für den Container und die “Tiefe” des Containers zu finden.

Die API von generic serialize ist ebenfalls sehr einfach und überschaubar. Sie funktioniert so:

namespace usb = util::serialize::binary;

/* Example: Write nested container to a binary file */
std::list<std::map<std::string, unsigned>>  list = {
  {{"foo", 1}, {"bar", 2}},
  {{"baz", 1}, {"quux", 2}}
};

/* save */
std::ofstream   os{"file.bin", std::ios::out | std::ios::binary};
usb::save(os, list);
os.close();

Das Einlesen von Binärdaten mit generic serialize ist ebenfalls sehr einfach:

namespace usb = util::serialize::binary;

/* Example: Read nested container from binary file, way #1 */
std::ifstream   is{"file.bin", std::ios::in | std::ios::binary};
std::list<std::map<std::string, unsigned>>  list;
usb::load(is, list);

Ein anderer Weg wäre es, den Container-Typ als template-Argument anzugeben und C++11’s auto zu nutzen:

namespace usb = util::serialize::binary;

/* Example: Read nested container from binary file, way #2 */
std::ifstream   is{"file.bin", std::ios::in | std::ios::binary};

auto list = usb::load<
    std::list<std::map<std::string, unsigned>>
  >(is);

Auf diese Art und Weise werden beliebige Container in einem Format wie in der Grafik ab Seite angedeutet, angelegt.

template<typename C>
struct is_container<C,
    typename is_of_type<
        typename C::const_iterator(C::*)() const,
        &C::begin,
        &C::end
    >::type
> : std::is_class<C> { };

17.4.1 Schritt 1: Einsteigsdatei addon.cpp schreiben

Eine kleine Datei addon.cpp wird die gesamte Struktur von C++ nach Javascript verbinden, und sieht – dem Beispiel der Dokumentation folgend –, so aus:

#ifndef BUILDING_NODE_EXTENSION
#define BUILDING_NODE_EXTENSION
#endif
#include <node.h>
#include "WLNodeWrap.hpp"

using namespace v8;

void InitAll(Handle<Object> exports) {
  WLNodeWrap::Init(exports);
}

NODE_MODULE(WLNodeWrap, InitAll)

Hier wird festgelegt, dass ein Modul mit dem Namen WLNodeWrap definiert wird, und, sobald dieses per require (var addon = require('./build/Release/WLNodeWrap'); auf Javascript-Seite aufgerufen wird, zur Funktion InitAll gesprungen werden soll.

Die Funktion InitAll wiederum wird WLNodeWrap::Init aufrufen, welches wir sogleich definieren werden.

17.4.2 Schritt 2: Wrapper-Klasse WLNodeWrap, die von node::ObjectWrap erbt, schreiben

Die Wrapper-Klasse WLNodeWrap wiederum orientiert sich sehr stark an der o.g. node-Dokumentation.

Im wesentlichen erbt sie von node::ObjectWrap und exportiert eine static-Methode Init. (Diese static Methode ruft der vorherige Schritt auf: NODE_MODULE -> WLNodeWrap -> InitAll -> WLNodeWrap::Init!)

Desweiteren wird die eigentliche, zu wrappende Klasse WL<std::string> der Klasse WLNodeWrap als Member hinzugefügt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

#ifndef NODEWRAP_HPP_SBNDJF7E
#define NODEWRAP_HPP_SBNDJF7E

#include <node.h>
#include <string>
#include "WL.hpp"

class WLNodeWrap : public node::ObjectWrap {
    typedef WL<std::string> wrapped_wl;
    
public:
    static void Init(v8::Handle<v8::Object> exports);

private:
    explicit WLNodeWrap() = default;
    virtual ~WLNodeWrap() = default;

    static v8::Handle<v8::Value> New(const v8::Arguments& args);
    
    // Add wird nach WL.add(const std::string&) gemapped werden
    static v8::Handle<v8::Value> Add(const v8::Arguments& args);
    // Size wird nach WL.size() gemapped werden
    static v8::Handle<v8::Value> Size(const v8::Arguments& args);
    static v8::Persistent<v8::Function> constructor;

    wrapped_wl wl_;
};

#endif /* end of include guard: NODEWRAP_HPP_SBNDJF7E */

Selbstverständlich gibt es eine passende Datei WLNodeWrap.cpp, welche diese Deklarationen des Header definiert, auch hier kann man sich wieder sehr eng an die node-Dokumentation halten:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

#ifndef BUILDING_NODE_EXTENSION
#define BUILDING_NODE_EXTENSION
#endif

#include <node.h>
#include "WLNodeWrap.hpp"

using namespace v8;

Persistent<Function> WLNodeWrap::constructor;

void WLNodeWrap::Init(Handle<Object> exports) {

  // Prepare constructor template
  Local<FunctionTemplate> tpl = FunctionTemplate::New(New);
  tpl->SetClassName(String::NewSymbol("WLNodeWrap"));
  tpl->InstanceTemplate()->SetInternalFieldCount(1);

  // Prototype
  tpl->PrototypeTemplate()->Set(String::NewSymbol("add"),
      FunctionTemplate::New(Add)->GetFunction()
  );
  tpl->PrototypeTemplate()->Set(String::NewSymbol("size"),
      FunctionTemplate::New(Size)->GetFunction()
  );

  constructor = Persistent<Function>::New(tpl->GetFunction());
  exports->Set(String::NewSymbol("WLNodeWrap"), constructor);
}

Handle<Value> WLNodeWrap::New(const Arguments& args) {
  HandleScope scope;

  if (args.IsConstructCall()) {
    // Invoked as constructor: `new WLNodeWrap(...)`
    WLNodeWrap* obj = new WLNodeWrap();
    obj->Wrap(args.This());
    return args.This();
  } else {
    // Invoked as plain function `WLNodeWrap(...)`, turn into construct call.
    const int argc = 1;
    Local<Value> argv[argc] = { args[0] };
    return scope.Close(constructor->NewInstance(argc, argv));
  }
}

Handle<Value> WLNodeWrap::Add(const v8::Arguments& args) {
    HandleScope scope;

    // Standard unpacking of strings
    Local<String>           String = args[0]->ToString();
    String::Utf8Value       ustring{ String };
    std::string             str{ *ustring };

    // Unwrap WL to `This'
    WLNodeWrap* This = node::ObjectWrap::Unwrap<WLNodeWrap>(args.This());

    // Talk to WL's member `wl_'
    This->wl_.add(str);
    
    return scope.Close(Undefined());
}

Handle<Value> WLNodeWrap::Size(const v8::Arguments& args) {
    HandleScope scope;

    // Unwrap WL to `This'
    WLNodeWrap* This = node::ObjectWrap::Unwrap<WLNodeWrap>(args.This());

    // Return WL::wl_'s return value to Javascript (as a number)
    return scope.Close( Number::New( This->wl_.size() ) );
}

17.4.3 Schritt 3: binding.gyp, die build-Datei für node-gyp schreiben

Nicht zuletzt muss noch eine Datei binding.gyp angelegt werden, welche – ganz wie ein Makefile –, die build-Datei für node-gyp darstellt.

node-gyp ist ein spezielles build-Tool (vergleiche Build Systeme) für “node-addons” und kümmert sich um das notwendige linking zu den benötigten node- und v8-libraries.

Zum vorhandenen Projekt sieht die binding.gyp folgendermaßen aus (es geht in dieser binding.gyp v.a. auch darum, C++11 transparent, über verschiedene Betriebssysteme hinweg verwenden zu können):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

{
  "targets": [
    {
      "target_name": "WLNodeWrap",
      "sources": [ "addon.cpp", "WLNodeWrap.cpp" ]
    }
  ],
  # global flags go here
  # `cflags_cc!` is supposed to mean: remove these flags
  # `cflags_cc` is supposed to mean: add these flags
  'conditions': [
      [
          'OS=="mac"', {
              'xcode_settings': {
                  'GCC_ENABLE_CPP_RTTI': 'YES',
                  'GCC_ENABLE_CPP_EXCEPTIONS': 'YES',
                  'CLANG_CXX_LANGUAGE_STANDARD': 'c++11',
                  'MACOSX_DEPLOYMENT_TARGET': '10.7',
                  'OTHER_CFLAGS':[ 
                    '-fPIC',
                    '-Wall',
                    '-Wextra',
                    '-Wno-attributes',
                    '-Wno-pointer-arith'
                  ]
              }
          }
      ],
      [
          'OS=="linux"', {
              'cflags_cc!': [ '-fno-rtti' ],
              'cflags_cc': [
                '-frtti',
                '-fPIC',
                '-fexceptions',
                '-std=c++11',
                '-Wall',
                '-Wextra',
                '-Wno-attributes',
                '-Wno-pointer-arith'
              ]
          }
      ]
  ]
}

Hiermit ist es also möglich, wie oben gezeigt, die C++-Klasse WL von Javascript aus anzusprechen:

1
2
3
4
5
6
7
8
9
10

var addon = require('./build/Release/WLNodeWrap');

var wl = new addon.WLNodeWrap();

wl.add('foo');
wl.add('bar');
wl.add('baz');

var size = wl.size();
console.log( size ); // 3

19.2.1 Unit-Tests

Ein Unit-Test bildet klassischerweise die Funktionalität einer Klasse in einer Isolierten Art und Weise ab.

Idealerweise bildet der Unit-Test zu einer Klasse zugleich auch ihre Spezifikation ab und prüft diese anhand der Fixtures durch.

class stringAppendFixture {
  string teststring1_append_in  = "foobar";
  string teststring1_append_out = "foobarc";
};

class stringAppendFeature {
  EXPECT(
       stringAppendFixture::teststring1_append_in.append('c')
    == stringAppendFixture::teststring1_append_out
  );
  // weitere tests der Spezifikation...
};

class stringAppendFixture {
  string teststring1_append_in  = "foobar";
  string teststring1_append_out = "foobarc";
  string teststring2_append_in  = "";
  string teststring2_append_out = "c";
};

class stringAppendFeature {
  EXPECT(
       stringAppendFixture::teststring1_append_in.append('c')
    == stringAppendFixture::teststring1_append_out
  );

  // hier gibt es schon den zweiten Testfall!
  EXPECT(
       stringAppendFixture::teststring2_append_in.append('c')
    == stringAppendFixture::teststring2_append_out
  );
  // weitere tests der Spezifikation...
};

class stringAppendFixture {
  std::map<string/*input*/, string /*output*/> fixture{
   { "foobar",    "foobarc"   },
   { "",          "c"         },
   { "foobar\n",  "foobarc\n" },
 };
};

class stringAppendFeature {
  const stringAppendFixture F;

  for (const auto & f: F.fixture) {
    EXPECT( f.first == f.second );
  }
};

19.2.2 Integration Tests

Integration Tests testen in den meisten Fällen das Zusammenspiel von einzelnen Klassen oder Software-Komponenten. Wie im vorherigen Abschnitt gezeigt wurde, sind diese einzelnen Klassen bereits durch die Unit-Tests in sich getestet. Der Integration Test nun testet diese einzelnen Kompomeneten in ihrem Zusammenhang, ihrer Kooperation etc. Diese Art von Tests wird aber zumeist eher ab einer umfangreicheren Größe des Projekts benötigt.

19.2.3 Sanity Tests

Neben Unit- und Integration-Tests gibt es auch noch die sog. Sanity-Tests. Diese Art von Tests validiert und prüft User-Eingaben, Datentypen und ähnliches auf korrekte Werte.

19.2.4 E2E Tests

E2E-Tests, bzw. End-to-end-tests sind sehr ähnlich zu Integration Tests, indem sie das Zusammenspiel der einzelnen Komponenten prüfen, haben aber eher ein konkretes Interaktions-Szenario des Benutzers im Blick: oft werden e2e Tests in Web-Frameworks und in der Entwicklung von Webseiten verwendet, um ein komplettes “Szenario” der Benutzer-Interaktion durchzuspielen und zu prüfen, ob dieses von Anfang bis Ende fehlerlos ablaufen kann.

Beispiel:

• User geht auf Seite
• User sucht etwas
• User klickt ein Ergebnis an
• User wird auf Seite des Ergebnisses geleitet
• User will dort eine Eingabe machen
• User muss angemeldet sein
• User meldet sich an
• User is angemeldet
• User kann Eingabe machen
• User kann Eingabe speichern
• …

20.2.1 Konventionen

Bugs sollen ordentlich dokumentiert werden und getagged werden.

Taggen erlaubt es, die offenen bugs zu sortieren, filtern und damit besser im Überblick behalten zu können.

Ordentlich dokumentieren bedeutet, dass ein nachfolgender genau nachvollziehen kann, wie/wann/wer mit einem Bug umgegangen wurde.

Dazu gehören:

• taggen: priorisieren (siehe unten, Prioritäten)
• taggen: Status (siehe unten, Workflow)
• dokumentieren: Wenn sich der Status eines Bugs ändert, am besten einen Kommentar im Bug-Verlauf hinterlassen, am besten so:
  • Beispiel: [Issued -> Analyze]. Dies würde bedeuten, dass sich Person Foo mit dem Bug auseinandersetzt und den tag Issued entfernt und durch den tag Analyze ersetzt hat. Dies bedeutet für alle anderen, dass:
    • sich bereits jemand um den Bug kümmert
    • andere können die noch offenen bugs bequem filtern und sehen, was als nächstes zu tun ist
  • Beispiel: [p1 -> p2].
    • Bedeutet: Person Foo hat den Bug herunterpriorisiert von P1 nach P2
    • Beispiel: [+p1]
      • Bedeutet: Person hat P1-tag an den Bug hinzugefügt

20.2.2 Prioritäten

Bugs sind am besten Priorisiert von “P1” bis “P4”.

Die Prioritäten entsprechen ungefähr folgenden Ideen:

20.2.3 Workflow

Der Workflow orientiert sich an folgendem Modell: Der Bug geht durch verschiedene Phasen, die unterschiedliche Bedeutung haben. Diese werden immer per tag signalisiert. So können andere ablesen, dass jemand an einem bug arbeitet. fasst dies zusammen.

Bug Tracking Workflow Status

20.2.4 Spezialität: Umbrella-Bug

Wenn es mehrere “related” Bugs zu einem größeren Thema gibt, so bietet es sich an einen umbrella-Bug zu definieren, der alle diese verstreuten, kleineren Bugs zusammenfasst und miteinander referenziert. Die wird am Einfachsten so gehandhabt, dass:

• der umbrella-Bug bekommt einen tag ‘umbrella’
• der umbrella bug bekommt eine Bezeichnung/Beschreibung ‘umbrella bug’
• der umbrella bug referenziert die die related-bugs, indem er tags bekommt mit den bug-id’s der related bugs, also, z.B. ‘#19, #22, #53’. Damit wäre klar, dass der umbrella-bug die bugs mit den Nummern 19, 22 und 53 zusammenfasst

Wenn man in der Beschreibung des Umbrella Bugs eine kleine Beschreibung anlegt, die sagt ‘this is an umbrella bug for bugs #19, #22, #53’, so wird in den referenzierten Bugs dieses automatisch von gitlab eingetragen: ‘this bug was mentioned in bug XY’. Dies ist ziemlich praktisch.

21.1.1 Vorteile von git

21.1.2 git basics

Um schneller mit git arbeiten zu können werden im folgenden die wesentlichen ersten Schritte detailliert erklärt und erläutert. Auf jeden Fall wird empfohlen, weitere Literatur zu diesen Themen zu konsultieren.

21.1.2.1 Überblick

Im Gegensatz zu svn gibt es bei git mehr Ebenen als nur “local repository” und “upstream repository”.

auf Seite gibt ein Überblick⁵⁹ über die unterschiedlichen Ebenen und die Interaktion zwischen denselben:

Git overview

> mkdir eos && cd $_

> cat <<HERE  >README
eos
---

an end-of-sentence recognizer
HERE

> git init

> git commit -am 'initial commit'

> git status

# On branch master
#
# Initial commit
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   README
nothing added to commit but untracked files present (use "git add" to track)

> git add README

> git status

# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#   new file:   README
#

> git commit -am 'add documentation'

> git status

# On branch master
nothing to commit (working directory clean)

commit 9112df662257116774d946cb5482116263cbc511
Author: AUTORENNAME <EMAILADRESSE></emailadresse>
Date:   DATUM

    add documentation

9112df

> git rev-parse 9112df
9112df662257116774d946cb5482116263cbc511

> git branch
* master

> git help branch
[...]
-a, --all
    List both remote-tracking branches and local branches.
[...]

> git branch -a
* master

> git branch dev

> git branch
dev
* master

> git checkout dev
Switched to branch 'dev'

> git branch
* dev
  master

> cat eos.pl
#!/usr/bin/env perl -0 -p
s/
 (
  (
   (
    (Mr|[[:upper:]]|\d+)\.  # Ausnahmen: Abkürzungen etc.
    |
    .+?                     # alles andere (non-greedy,
                            #  d.h. 1. nicht [.!?] gefolgt von SPACE + Großbuchstabe)
                            #       2. keine Abkürzung
   )+?                      # eine Folge von Abkürzungen oder "alles"
   [.!?](?=\s+[[:upper:]])  # EOS
  )
 )
/$1<EOS>/gx;
EOS

> cat eos.t
Ein Satz? Hier auch! Am 19. August 1972 schläft
Mr. G. W. Spencer. Hier beginnt ein Satz.  In der
Oettingenstr. 67 stehen Computer.
Perl arbeitet hier zeilenweise, Sätze dürfen nicht über Zeilen gehen.
TXT

> chmod u+x eos.pl

> mkdir var

> cat eos.t | ./eos.pl > var/res.1

> git status
# On branch dev
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   eos.pl
#   eos.t
#   var/

echo var/ >> .gitignore

git status
# On branch dev
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   .gitignore
#   eos.pl
#   eos.t

> git add .gitignore

> git commit -am 'add gitignore'
[dev 1f986bd] add gitignore
 1 file changed, 1 insertion(+)
 create mode 100644 .gitignore

> git status
# On branch dev
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   eos.pl
#   eos.t
nothing added to commit but untracked files present (use "git add" to track)


> git add eos.*

> git commit -am 'add first version of eos + test data'
[dev 46b34ab] add first version of eos + test data
 2 files changed, 19 insertions(+)
 create mode 100755 eos.pl
 create mode 100644 eos.t

> git status
# On branch dev
nothing to commit (working directory clean)

> git log --oneline --decorate --graph
* 46b34ab (HEAD, dev) add first version of eos + test data
* 1f986bd add gitignore
* 9112df6 (master) add documentation

> git branch
* dev
  master

> ls -1
README
eos.pl
eos.t
var

> git checkout master
Switched to branch 'master'
> ls -1
README
var

> git checkout dev
Switched to branch 'dev'
> ls -1
README
eos.pl
eos.t
var

> git remote -v 
[kein output]

> git remote add origin name@machine:path/to/bare/repository.git
> git remote -v
origin  name@machine:path/to/bare/repository.git (fetch)
origin  name@machine:path/to/bare/repository.git (push)

> git push origin master

> git pull origin master

21.1.3 git-extras

git-extras (visionmedia 2014) geben viele nützliche (neue) Kommandos zu den Standard git-Kommandos hinzu.

Dadurch kürzen sie einige Wege ab, oder erleichtern andere, routine-mäßige Aufgaben.

Da sie aber mit Hilfe der von git selbst zur Verfügung gestellten Kommandos implementiert sind, könnte man selbstverständlich alles auch ohne git-extras machen.

Dennoch können die git-extras als eine sinnvolle Empfehlung betrachtet werden.

git clone https://github.com/visionmedia/git-extras
cd git-extras
make DESTDIR=$HOME PREFIX="" install

pltk@master$              git feature lemmatizer
pltk@feature/lemmatizer$  [hack hack (möglichst kleine Commits bitte!)]
pltk@feature/lemmatizer$  git commit -am 'last commit'
pltk@feature/lemmatizer$  git checkout master
pltk@master$              git feature finish lemmatizer

# [In gitlab ist ein issue im bugtracker eröffnet]
pltk@master$        git bug issue-66
pltk@bug/issue-66$  [ hack hack ...]   
pltk@bug/issue-66$  [ letzter commit ]
pltk@bug/issue-66$  git checkout master
pltk@master$        git feature finish issue-66

22.1.1 Ubuntu-Schema

Das Versionsschema von WAST orientiert sich nach außen am Versions-Schema von Ubuntu. Dabei wrid die Versionsnummer zusammengesetzt aus der Jahreszahl und dem Monat des Releases.

Im oben gegebenen Beispiel hat sich X mit v14.02 auf die Version von WAST zum Zeitpunkt Februar 2014 bezogen. Die Version von November 2015 wird also 15.11 sein. Dabei ist ein monatlicher Release-Cycle gegeben.

Wie gesagt, dies ist die Versionierung nach außen, d.h. im Zusammenspiel aller Komponenten. In der Versionierung nach innen sollte sich jede der Komponenten selbst jedoch am Besten am Schema des Semantic Versioning orientieren.

Dementsprechend kann ein Release und die Version desselben genauer unterteilt werden:

Beispiel:

• Release 14.02 könnte nämlich in diesem Sinne eine Zusammenstellung der folgenden Komponenten gewesen sein:
  • WAB-Transkription 2.3.1
  • wittfind-Komponente 4.2.3
  • sis-Komponente 3.9.1
  • …

22.1.2 Semantic Versioning

Im wewsentlichen versteht man unter semantic versioning eine Versionsangabe mit folgendem Format: Major.Minor.Patch.

Dabei stehen Major-Versionen für (z.T.) untereinander nicht-kompatible APIs oder große Veränderungen/Verbesserungen am Code, bzw. Verweisen auf wesentliche Entwicklungsschritte im Code.

Die Zahl unter Minor bezeichnet den aktuellen Entwicklungsstand des Codes. In den meisten Fällen werden als Minor ungerade Zahlen verwendet um anzudeuten, dass dies eine Entwickler-Version ist, und, nachdem diese Entwicklung eines bestimmten Features (oder ähnlichem) abgeschlossen ist, diese dann um eins hochgezählt zu einer geraden Zahl.

Nicht zuletzt wird Patch bei bugfixes hochgezählt.

Vergleiche z.B.

• Perl, Version 5.19.4 (Perl5, Weiterentwicklung von 5.18 (als Entwicklerversion), patch-level 4)
• nodejs, Version 0.10.20 (Noch nicht als 1.0.0-fähig anerkanntes, d.h. noch nicht komplett stabilisiertes, dennoch fortgeschrittenes Projekt in einer 0.10-er-“stable version” mit patch-level 20)

Mehr Informationen zu Sematic Versioning finden sich bei http://semver.org/(Tom Preston-Werner 2013)

23.1.1 GNU GPLv2

23.1.2 GNU GPLv3

23.1.3 GNU LGPL

23.1.4 GPL-Compaitibility

23.1.5 Affero GPL

23.1.6 GPLv2 Loophole

23.3.1 CC-Bausteine

23.3.2 NC-BY-SA

23.3.3 NC-BY …

28.1.1 Komponenten, Beschreibungen, Maintainer

Component:     wf
Description:
Maintainer:    flo@cis.lmu.de
Link external: wittfind.cis.lmu.de
Repository:    gitlab.cis.lmu.de/wast/wf
Dependencies:
    * `gcc > 4.7`
    * boost
        * ... (?)
    * cmake
    * (doxygen)

Component:      wittfind-web     
Description:
Maintainer:     maximilian@cis.lmu.de et al.
Link external:  wittfind.cis.lmu.de
Repository:     gitlab.cis.lmu.de/wast/wittfind-web
Dependencies:
    * git
        * git-submodules
            * wf
    * PHP
    * Apache
    * E2E-dependencies:
        * [^node] / [^npm]
            * [^casperjs]
                * [^phantomjs]

Component:      SIS (frontend)
Description:
Maintainer:     bruder.cis.lmu.de
Link external:  sis.cis.lmu.de
Repository:     gitlab.cis.lmu.de/wast/sis3/tree/master/web
Dependencies:   
    * MEAN-Stack
        * [^node] / [^npm]
        * [^expressjs]
        * [^angularjs]
        * [^bower]
            * client-seitige dependencies
              werden durch [^bower]
              installiert
        * [^npm]
            * server-seitige dependencies
              werden durch [^npm]
              installiert
        * MongoDB

Component:      SIS (backend)
Description:
Maintainer:     bruder@cis.lmu.de
Link external:  sis.cis.lmu.de
Repository:     gitlab.cis.lmu.de/wast/sis3/tree/master/src   
Dependencies:
    * `gcc > 4.7`
    * cmake
    * (doxygen)
    * git
        * git-submodules
            * lest
    * boost
        * ... (?)

Component:      "Alternativen-Suche"
Description:    Ausformulierung von Alternativen
Maintainer:     seebauerp@cip.ifi.lmu.de / lindinger@cip.ifi.lmu.de / TBD
Link external:  wittfind.cis.lmu.de
Repository:     gitlab.cis.lmu.de/wast/wittfind-web   
Dependencies:
    * perl-module (?)
        * XML-Twig (?)
        * ... (?)
    * ... (?)

Component:      Helppage
Description:    Beispiel-Anfragen und Hilfe für wittfind
Maintainer:     kreya@cip.ifi.lmu.de
Link external:  wittfind.cis.lmu.de/help.php
Repository:     gitlab.cis.lmu.de/wast/wittfind-web
Dependencies:
    * PHP (Version?)
        * ... (?)
    * ... (?)

Component:      Feedback
Description:    Bug Reports für Aussenstehende
Maintainer:     bruder@cis.lmu.de, TBD
Link external:  wastfeedback.cis.lmu.de
Repository:     gitlab.cis.lmu.de/wast/wast-feedback
Dependencies:
    * curl
    * gitlab user token / "FeedbackBot"-User
    * MEAN-Stack
        * [^node] / [^npm]
        * [^expressjs]
        * [^angularjs]
        * [^bower]
            * client-seitige dependencies
              werden durch [^bower]
              installiert
        * [^npm]
            * server-seitige dependencies
              werden durch [^npm]
              installiert
        * MongoDB

Component:      E2E
Description:    End-to-end tests für wittfind-web: komplette User-Szenarien durchspielen
Maintainer:     bruder@cis.lmu.de, TBD
Link external:  ---
Repository:     gitlab.cis.lmu.de/wast/wittfind-web/tree/development/tests
Dependencies:
    * [^node] / [^npm]
        * [^casperjs]
            * [^phantomjs]

Component:      Reader
Description:    Reader-Applikation für Facsimile
Maintainer:     lindinger@cip.ifi.lmu.de (et al.?)
Link external:  ---
Repository:     gitlab.cis.lmu.de/wast/wittfind-web/tree/development/tests
Dependencies:
    * PHP (Version?)
    * [^turn.js]
    * ... (?)

Component:      wast-doc
Description:    Dokumentation für WAST                                                                                      
Maintainer:     bruder@cis.lmu.de / TBD
Link external:  www.cip.ifi.lmu.de/~bruder/wast/
Repository:     https://gitlab.cis.lmu.de/wast/wast-doc
Dependencies:
    * [make](https://www.gnu.org/software/make/manual/make.html)
    * [pandoc](johnmacfarlane.net/pandoc/)
    * [gpp](http://files.nothingisreal.com/software/gpp/gpp.html)
    * [/github.com/typesetters/p5-App-pandoc-preprocess)
        * [dot](http://www.graphviz.org/Documentation.php)/[neato](http://www.graphviz.org/pdf/neatoguide.pdf)
        * [rdfdot](RDF::Trine::Exporter::GraphViz)
        * [ditaa](http://ditaa.sourceforge.net/)
        * [Image::Magick](http://www.imagemagick.org/) (image manipulation)
        * [rsvg-convert](http://live.gnome.org/LibRsvg)
        * [yuml](https://github.com/wandernauta/yuml) (install as python module using `pip install https://github.com/wandernauta/yuml/zipball/master` or `easy_install https://github.com/wandernauta/yuml/zipball/master`)
        * [plantuml](http://plantuml.sourceforge.net/)

28.1.2 Links

[^wf]:              <https://gitlab.cis.lmu.de/wast/wf>
[^SIS-backend]:     <https://gitlab.cis.lmu.de/wast/sis3/tree/master/src>
[^SIS-frontend]:    <https://gitlab.cis.lmu.de/wast/sis3/tree/master/web>
[^wittfind-web]:    <https://gitlab.cis.lmu.de/wast/wittfind-web>
[^feedback]:        <https://gitlab.cis.lmu.de/wast/wast-feedback>
[^feedback-link]:   <http://wastfeedback.cis.uni-muenchen.de/>
[^e2e]:             <https://gitlab.cis.lmu.de/wast/wittfind-web/tree/development/tests>
[^reader]:          <https://gitlab.cis.lmu.de/wast/wittfind-web/>
[^cmake]:           <http://www.cmake.org/>
[^lest]:            <https://github.com/martinmoene/lest>
[^node]:            <http://nodejs.org/>
[^npm]:             <http://nodejs.org/>
[^expressjs]:       <http://expressjs.com/>
[^angularjs]:       <https://angularjs.org/>
[^bower]:           <http://bower.io/>
[^phantomjs]:       <http://phantomjs.org/>
[^casperjs]:        <http://casperjs.org/>
[^turn.js]:         <http://www.turnjs.com/>
[^doxygen]:         <http://doxygen.org/>
[^sis-link]:        <http://sis.cis.lmu.de/>
[^wf-link]:         <http://wittfind.cis.lmu.de/>
[^wittfind-link]:   <http://wittfind.cis.lmu.de/>
[^helppage]:        <https://gitlab.cis.lmu.de/wast/wittfind-web>

28.1.3 Dependency Graph (experimental)

WAST Komponenten und Dependencies