Packages

package root

Definition Classes: root

package ch

Definition Classes: root

package usi

Definition Classes: ch

package inf

Definition Classes: usi

package reveal

Definition Classes: inf

package parsing

Definition Classes: reveal

package artifact

Provides classes for dealing with Stack Overflow artifacts.

It also provides means to serialize and deserialize them from JSON files.

Tutorial

The main class to consider is ch.usi.inf.reveal.parsing.artifact.StackOverflowArtifact, representing a Stack Overflow discussion.

The main way to obtain a Stack Overflow discussion is to deserialize it froma JSON file, using ArtifactSerializer.

import ch.usi.inf.reveal.parsing.artifact._
val jsonFilePath = /*path to json file*/
val serializer = new ArtifactSerializer()
val discussion = serializer.deserializeFromFile(jsonFilePath)

You can now inspect the discussion, for example by getting the question (a StackOverflowQuestion object) or answers (a seq of StackOverflowAnswer):

scala> val question = discussion.question
question: ch.usi.inf.reveal.parsing.artifact.StackOverflowQuestion = ...

scala> val answers = discussion.answers
answers: Seq[ch.usi.inf.reveal.parsing.artifact.StackOverflowAnswer] = ...

The way to get the object representing the poster of the question (a StackOverflowUser) is the following:

scala> val posterUser = question.owner
posterUser: Option[ch.usi.inf.reveal.parsing.artifact.StackOverflowUser] = Some(StackOverflowUser(...))

You can also easily get the comments (objects of StackOverflowComment), for example the ones posted for the question:

scala> val questionComments = question.comments
questionComments: Seq[ch.usi.inf.reveal.parsing.artifact.StackOverflowComment] = ...

The easy way to get all information units (ch.usi.inf.reveal.parsing.units.InformationUnit) and the HASTs (classes implementing the ch.usi.inf.reveal.parsing.model.HASTNode trait) for an element (question or answer or comment) is:

scala> val allUnits = discussion.units
allUnits: Seq[ch.usi.inf.reveal.parsing.units.InformationUnit] = List(...)

scala> val astNodes = allUnits.map { _.astNode }
astNodes: Seq[ch.usi.inf.reveal.parsing.model.ASTNode] =

For further information about the information units, the HAST and the model, please check the documentation of ch.usi.inf.reveal.parsing.model and ch.usi.inf.reveal.parsing.units.

Definition Classes: parsing

package model

Provides base classes to model HAST nodes for Stack Overflow artifacts.

Overview

Every note of the HAST implements the HASTNode trait.

Depending on the specific fragment, nodes implement traits in different packages. For example:

ch.usi.inf.reveal.parsing.model.java;
ch.usi.inf.reveal.parsing.model.json;
ch.usi.inf.reveal.parsing.model.stacktraces;
ch.usi.inf.reveal.parsing.model.xml;
and the package for visitors, ch.usi.inf.reveal.parsing.model.visitors.

Special nodes are used for various purposes, and are contained in this package:

HASTNodeSequence for sequences of structured fragments;
CommentNode for comments, which are special nodes in the HAST;
TextFragmentNode for textual (natural language narrative) fragments.

Definition Classes: parsing

package common

package java

Contains the model classes for Java fragments.

Overview

The main trait to identify Java fragments is JavaASTNode.

Examples of interesting nodes in this package are:

nodes for compilation units (CompilationUnitNode), classes (ClassDeclarationNode), interfaces (InterfaceDeclarationNode), etc.;
method declarations (MethodDeclarationNode), fields (FieldDeclarationNode), etc.;
expressions (e.g., BinaryExpressionNode, PrefixOperatorExpressionNode, CastExpressionNode) and operators (OperatorNode);
structured programming constructs (e.g. ForLoopStatementNode, IfStatementNode, etc.);
types (ReferenceTypeNode, PrimitiveTypeNode) and identifiers (e.g. QualifiedIdentifierNode, IdentifierNode);
other terminals (KeywordModifierNode, KeywordNode, etc.)

package json

Contains the model classes for JSON fragments.

Overview

Examples of interesting nodes in this package are:

JsonObjectNode, modeling json objects;
JsonMemberNode, modeling a name - value pair;
literals, like JsonStringNode and JsonBooleanNode.

package stacktraces

Contains the model classes for stack traces.

Overview

Examples of interesting nodes are:

StackTraceNode which represent a full stacktrace;
StackTraceLineNode which represents a single line of a stacktrace.

package transform

package visitors

Contains the visitors for HAST nodes.

See also: ArtifactVisitor

package xml

Contains the model classes for XML/HTML tags.

Overview

* Examples of interesting nodes are:

XmlSingleNode which represents a singleton xml/html node;
XmlComposedNode which represents a composed xml/html node;
XmlCommentNode which represents an xml/html comment.

CommentNode

EmbeddedContentsNode

EmbeddedContentsProvider

GenericEmbeddedContentsNode

HASTNode

HASTNodeSequence

Implicits

IslandParserResult

JsonSerializable

PTIdentifierNode

PTKeywordNode

PTLiteralNode

PTModifierNode

PTOperatorNode

PTSeparatorNode

PTWaterNode

ParseTreeInternalNode

ParseTreeNode

ParseTreeToCodeImplementation

StructuredFragmentResult

TestingRepresentation

TextFragmentNode

TextFragmentResult

Tree

WhiteSpaceNode

package nlp

Definition Classes: parsing

package units

Provides classes representing information units (structured and textual units) and meta information (like tf vectors and type mentions).

Overview

Information units (implementing the trait InformationUnit) represent paragraphs in a given document, which can be narrative text (NaturalLanguageTaggedUnit) or structured fragments (CodeTaggedUnit). Each information unit exports a set of meta-information (implementing the MetaInformation trait), which are ready made semantic data for simple analyses. This version of StORMeD provides the following meta-information:

NaturalLanguageMetaInformation for a natural language view of the information unit (can be used also for code);
CodeTypesMetaInformation providing the mentioned types in the unit;
CodeVariablesMetaInformation providing the mentioned variables;
CodeInvocationMetaInformation providing the invoked methods in the unit.

In the case a meta information is not provided for a unit, a AbsentMetaInformation object can be also provided.

Tutorial

Suppose you want to get all the types mentioned in a question. First, you retrieve all its information units:

scala> val questionUnits = question.units
questionUnits: Seq[ch.usi.inf.reveal.parsing.units.InformationUnit] = ...

Instead of using the visitor to on all the HASTs, you can exploit the ready made data provided by the meta information. For example, to get all the meta information for units, you can use flatMap:

scala> import ch.usi.inf.reveal.parsing.units._
import ch.usi.inf.reveal.parsing.units._

scala> val questionMetaInfos = questionUnits.flatMap { _.metaInformation }
questionMetaInfos: Seq[ch.usi.inf.reveal.parsing.units.MetaInformation] = List(...)

You need now to filter to get only the CodeTypesMetaInformation, and then you can get, for example, the mentioned qualified Types:

scala> val codeTypesMetaInfos = questionMetaInfos.filter { _.isInstanceOf[CodeTypesMetaInformation] }.asInstanceOf[Seq[CodeTypesMetaInformation]]
codeTypesMetaInfos: Seq[ch.usi.inf.reveal.parsing.units.CodeTypesMetaInformation] = List(...)

scala> val types = codeTypesMetaInfos.flatMap { _.qualifiedTypes }.distinct
types: Seq[ch.usi.inf.reveal.parsing.model.java.ReferenceTypeNode] = List(...)

Definition Classes: parsing

ch.usi.inf.reveal.parsing

model

package model

Provides base classes to model HAST nodes for Stack Overflow artifacts.

Overview

Every note of the HAST implements the HASTNode trait.

Depending on the specific fragment, nodes implement traits in different packages. For example:

ch.usi.inf.reveal.parsing.model.java;
ch.usi.inf.reveal.parsing.model.json;
ch.usi.inf.reveal.parsing.model.stacktraces;
ch.usi.inf.reveal.parsing.model.xml;
and the package for visitors, ch.usi.inf.reveal.parsing.model.visitors.

Special nodes are used for various purposes, and are contained in this package:

HASTNodeSequence for sequences of structured fragments;
CommentNode for comments, which are special nodes in the HAST;
TextFragmentNode for textual (natural language narrative) fragments.

Linear Supertypes

AnyRef, Any

Ordering

Alphabetic
By Inheritance

Inherited

model
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Type Members

trait CommentNode extends ParseTreeNode with EmbeddedContentsProvider
A comment node.
A comment node.
This common trait is used to hold comments, which are not normally considered as HAST nodes, but they are included for additional analyses (e.g. the ones involving documentation or natural language). It may provide embedded contents, e.g., documentation contents, or unstructured data.
trait EmbeddedContentsNode extends HASTNode
Represents the embedded contents of a node, that is, a sequence of H-AST nodes representing the structured contents of the node itself.
trait EmbeddedContentsProvider extends AnyRef
Represents a H-AST node that optionally contains some embedded contents.
case class GenericEmbeddedContentsNode (nodes: Seq[HASTNode]) extends HASTNode with EmbeddedContentsNode with Product with Serializable
A generic implementation for embedded contents, where each node can be any type of HAST node.
trait HASTNode extends Product with JsonSerializable
A node of the heterogeneous AST node.
case class HASTNodeSequence (fragments: Seq[HASTNode]) extends HASTNode with Product with Serializable
A simple HAST node representing a sequence of fragments.
A simple HAST node representing a sequence of fragments.
fragments
a sequence of HAST nodes.
trait IslandParserResult extends AnyRef
A generic trait for results of Island Parsing.
trait JsonSerializable extends AnyRef
A tagger trait for case classes which can be serialized to JSON.
case class PTIdentifierNode (valueRep: String) extends ParseTreeNode with ParseTreeToCodeImplementation with Product with Serializable
A parse tree terminal node for identifiers.
case class PTKeywordNode (valueRep: String) extends ParseTreeNode with ParseTreeToCodeImplementation with Product with Serializable
A parse tree terminal node for keywords.
case class PTLiteralNode (valueRep: String) extends ParseTreeNode with ParseTreeToCodeImplementation with Product with Serializable
A parse tree terminal node for literals.
case class PTModifierNode (valueRep: String) extends ParseTreeNode with ParseTreeToCodeImplementation with Product with Serializable
A parse tree terminal node for modifier keywords (e.g., public).
case class PTOperatorNode (valueRep: String) extends ParseTreeNode with ParseTreeToCodeImplementation with Product with Serializable
A parse tree terminal node for operators.
case class PTSeparatorNode (valueRep: String) extends ParseTreeNode with ParseTreeToCodeImplementation with Product with Serializable
A parse tree terminal node for separators (e.g., commas).
case class PTWaterNode (valueRep: String) extends ParseTreeNode with ParseTreeToCodeImplementation with Product with Serializable
A parse tree terminal node that represents water in the island languages.
case class ParseTreeInternalNode (children: Seq[ParseTreeNode]) extends ParseTreeNode with Product with Serializable
An internal node of the parse tree (i.e., a node with children).
An internal node of the parse tree (i.e., a node with children).
children
the children of the internal node.
trait ParseTreeNode extends AnyRef
A node in the parse tree.
trait ParseTreeToCodeImplementation extends AnyRef
A trait providing a default implementation for terminal nodes in the parse tree.
case class StructuredFragmentResult (tree: Tree[HASTNode]) extends IslandParserResult with Product with Serializable
A structured fragment result, that is, a parsing result that contains some fragment of a structured language (e.g.
A structured fragment result, that is, a parsing result that contains some fragment of a structured language (e.g. code or markup languages like JSON).
trait TestingRepresentation extends AnyRef
Common trait for code conversion, generally used for testing purposes.
case class TextFragmentNode (text: String) extends ASTTerminalNode with BlockStatementNode with MemberDeclarationNode with JavadocASTNode with Product with Serializable
A text fragment node.
A text fragment node.
To support island with lakes, this is also a block statement and a member declaration node, but the corresponding methods have no effect.
text
the text of the fragment
case class TextFragmentResult (tree: Tree[TextFragmentNode]) extends IslandParserResult with Product with Serializable
A text fragment result, that is, a parsing result that mainly contains a textual fragment.
case class Tree [+NodeType <: HASTNode](ast: NodeType, parseTree: ParseTreeNode) extends Product with Serializable
Represents a combination of an HAST of a given type and the corresponding parse tree.
Represents a combination of an HAST of a given type and the corresponding parse tree.
NodeType
The type of the HAST.
case class WhiteSpaceNode (text: String) extends ParseTreeNode with Product with Serializable
A parse tree terminal node representing white space.

Value Members

object Implicits
If imported, allows to generate code from a HASTNode.
If imported, allows to generate code from a HASTNode. It automatically includes and manages implicit conversions for ch.usi.inf.reveal.parsing.model.json.JsonASTNode, ch.usi.inf.reveal.parsing.model.java.JavaASTNode, ch.usi.inf.reveal.parsing.model.xml.XmlASTNode, TextFragmentNode, and ch.usi.inf.reveal.parsing.model.stacktraces.StackTraceASTNode.

Packages

Tutorial

Overview

Overview

Overview

Overview

Overview

Overview

Tutorial

model

package model

Overview

Type Members

Value Members

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

Tutorial

Overview

Overview

Overview

Overview

Overview

Overview

Tutorial

model 

package model

Overview

Type Members

Value Members

Inherited from AnyRef

Inherited from Any

Ungrouped

model